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This  dissertation  concerns  empirical  Bayes  estimation  of  several  strata  means, 
where  each  stratum  contains  a  fixed  finite  number  of  elements.  The  results  of  this 
dissertation  will  find  their  application  in  small  area  estimation  where  very  few  obser- 
vations are  available  for  most  of  the  individual  areas.  The  smallness  of  the  sample 
size,  for  example,  may  be  due  to  the  fact  that  the  original  survey  was  targeted  to 
achieve  accuracy  at  a  much  higher  level  of  aggregation  than  that  of  local  areas.  Thus 
direct  estimates  of  the  local  area  means  are  less  precise.  Empirical  Bayes  methods  al- 
low one  to  borrow  strength  from  similar  neighboring  areas,  resulting  thereby  in  more 
reliable  estimates. 

The  observations  in  each  stratum  are  considered  to  be  realizations  from  a  super- 
population  of  one  parameter  exponential  family  distributions.  Conjugate  priors  are 
imposed  on  the  superpopulation  parameters  to  carry  out  an  empirical  Bayes  analy- 
sis. An  estimating  function  approach  is  used  to  estimate  the  prior  parameters,  often 
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referred  to  as  hyper-parameters.  Asymptotic  optimality  properties  of  the  resulting 
empirical  Bayes  estimates  of  the  stratum  means  are  also  proved.  The  developed 
methodology  is  applied  to  estimate  the  true  risk  of  cognitive  impairment  for  different 
principal  life  time  categories  for  an  elderly  cohort  in  France. 

A  major  problem  in  empirical  Bayes  analysis  is  to  correctly  calculate  the  standard 
errors  of  the  empirical  Bayes  estimates.  This  dissertation  also  provides  methodology 
to  compute  conditional  (on  ancillary  statistic  or  some  suitable  data  summary)  and 
unconditional  standard  errors  of  the  empirical  Bayes  estimates  by  using  parametric 
bootstrap  method.  Mathematical  results  are  proved  to  compare  the  conditional  and 
unconditional  asymptotic  distributions  of  the  standardized  EB  estimators.  Bootstrap 
methods  are  applied  to  obtain  conditional  and  unconditional  standard  errors  of  the 
empirical  Bayes  estimates  of  different  occupational  categories  in  the  cognitive  im- 
pairment problem.  Asymptotic  methods  are  used  to  develop  rules  of  thumb  for  the 
number  of  resamples  required  for  accurate  simultaneous  Monte  Carlo  approximation 
of  bootstrap  standard  errors. 
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CHAPTER  1 


INTRODUCTION 

1.1    The  Literature  Review 

Empirical  Bayes  (EB)  methodology  is  gaining  rapid  popularity  in  finite  population 
sampling.  This  can  partly  be  attributed  to  its  immediate  applicability  to  problems 
of  small  area  estimation.  In  these  problems,  one  is  typically  confronted  with  a  large 
number  of  local  areas  or  domains  with  relatively  small  sample  sizes  in  most  of  the 
individual  areas.  The  smallness  of  the  sample  size  is  due  to  the  fact  that  the  original 
survey  was  targeted  to  achieve  accuracy  at  a  much  higher  level  of  aggregation  than 
that  of  local  areas.  As  a  consequence,  direct  estimates  of  local  area  means  or  other 
quantities  of  interest  are  usually  accompanied  by  large  standard  errors  or  coefficients 
of  variation.  EB  methods  allow  one  to  borrow  strength  from  similar  neighboring 
areas,  resulting  in  more  reliable  estimates. 

Ghosh  and  Meeden  (1986)  considered  EB  estimation  of  the  finite  population  mean 
assuming  a  normal  superpopulation  model.  The  normality  assumption  was  later 
relaxed  by  Ghosh  and  Lahiri  (1987).  Instead,  it  was  assumed  there  that  the  posterior 
expectation  of  any  local  area  mean  was  a  linear  function  of  the  sample  observations. 
This  property  was  referred  to  as  posterior  linearity.  The  work  of  Ghosh  and  Meeden 
(1986)  was  later  generalized  by  Nandram  and  Sedransk  (1993). 

Normal  theory  occupies  a  central  role  in  much  of  the  existing  EB  literature  in 
finite  population  sampling.  One  of  the  most  well  cited  applications  of  normal  theory 
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EB  model  is  due  to  Fay  and  Herriot  (1979).  However,  more  recently  work  has  started 
on  the  EB  analysis  of  binary  survey  data.  For  small  area  estimates  of  proportions  via 
the  EB  technique,  one  may  refer  to  McGibbon  and  Tomberlin  (1989),  and  Farrell, 
McGibbon  and  Tomberlin  (1994). 

Closely  related  to  the  EB  method  is  the  hierarchical  Bayes  (HB)  method  which 
is  sometimes  more  effective  in  "borrowing  strength."  This  is  especially  so  when  one 
is  interested  in  the  standard  errors  in  addition  to  the  point  estimates  of  the  stratum 
means.  For  the  normal  theory  HB  analysis  in  the  small  area  context,  one  may  refer 
to  Datta  and  Ghosh  (1991),  Ghosh  and  Lahiri  (1992),  Ghosh  and  Rao  (1994)  among 
others.  Stroud  (1991)  develops  a  general  HB  methodology  for  binary  data,  while 
Nandram  and  Sedransk  (1993)  suggest  Bayesian  predictive  inference  for  binary  data 
from  a  two  stage  cluster  sample.  Melee,  Sedransk  and  Tomkins  (1994)  find  the  pre- 
dictive distributions  of  a  linear  combination  of  binary  random  variables  using  a  HB 
technique,  while  Stroud  (1994)  provides  a  comprehensive  treatment  of  binary  survey 
data  encompassing  simple  random,  stratified,  cluster  and  two  stage  sampling  within 
strata.  In  spite  of  the  success  of  HB  methodology,  the  EB  analysis  has  its  virtue,  in 
as  much  as  it  does  not  require  the  sophisticated  Monte-Carlo  numerical  integration 
techniques  which  are  currently  in  vogue  today.  In  addition  the  asymptotic  optimality 
results  proved  in  the  second  chapter  of  this  dissertation  for  the  EB  estimators  are  not 
yet  available  for  the  HB  estimators  even  for  simpler  models  based  on  normality. 

Though  the  EB  estimates  for  the  stratum  means  perform  very  well  asymptotically, 
the  naive  standard  errors  are  typically  too  small.  To  understand  this  problem  better 
we  introduce  the  EB  modeling  paradigm  as  defined  by  Morris  (1983a,b).  An  Empirical 
Bayes  model  is  a  collection  of  joint  probability  distributions  for  the  parameters  fj,  and 
the  data  y,  P  =  {pa(/j.,y),a  6  A},  say,  where  A  is  the  parameter  space.  Table 
1.1  contains  the  EB  modeling  paradigm  as  suggested  by  Morris.  The  paradigm  looks 
at  the  EB  model  from  two  view  points:  descriptive  and  inferential.  The  descriptive 
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viewpoint  is  how  EB  models  are  denned  and  the  inferential  viewpoint  is  how  they  are 
used. 


Table  1.1:  The  Empirical  Bayes  Modeling  Paradigm 


Descriptive 
Viewpoint 

Inferential 
Viewpoint 

Data 

{p(ylf)>/*€  0} 

(sampling  family) 

{pQ{y),ae  A} 
(marginal  family) 

Parameters 

{Po(m)>  a  e  A) 
(prior  family) 

{p(n\y,a),ae  A} 
(posterior  family) 

If  a  were  known  then  the  point  or  interval  estimates  for  \x  would  be  computed 
via  the  posterior  {p(n\y ,  ot) ,  a  6  A}.  Generally,  however,  a  is  unknown.  A  pure 
Bayesian  approach  would  place  a  second  stage  prior  (known  as  a  'hyperprior')  r(a) 
on  a  and  then  base  inference  about  /z  on  the  'marginal  posterior' 

Pr(Mltf)  =  /  p(n\v,a)pa(y)T(a)da. 

However,  in  the  EB  approach,  one  treats  a  as  a  fixed  unknown  and  replaces  it  with  an 
estimate  of  a  from  the  marginal  distribution  pa(y),  obtaining  d  =  dc(y).  Inference 
is  based  on  the  estimated  posterior  p(n\y,  a). 

Suppose  we  are  interested  in  estimating  g(n).  There  is  a  substantial  amount  of 
literature  which  demonstrates  that,  as  an  estimator  of  g{n),  E[g(fi)\y,  a]  performs 
well  in  a  decision  theoretic  sense  (Morris  1983a).  But  interval  estimation  of  /x  through 
the  estimated  posterior  distribution  is  less  successful.  Such  'naive'  confidence  intervals 
are  typically  too  short  and  hence  fail  to  attain  the  true  coverage  probability.  The 
reason  behind  this  is  that  here  we  are  ignoring  the  variability  in  d  or  in  other  words, 
from  a  Bayesian  point  of  view,  we  are  ignoring  the  posterior  uncertainty  about  a. 
More  precisely 

var(n\y)  =  Ea\y[var(fi\y,  a)}  +  vara\y[E(n\y,  or)}, 


and  so  the  variance  computed  on  the  basis  of  the  estimated  posterior  distribution, 
var(fx\y,a),  will  only  approximate  the  first  term  in  the  above  expression.  By  using 
a  full  Bayesian  model,  one  can  calculate  this  above  conditional  variance,  but  that 
expression  will  be  dependent  on  the  selection  of  prior  distribution  r(a).  In  an  EB 
model  as  defined  in  the  above  table,  one  can  try  to  estimate  the  unconditional  mean 
square  error,  E[(g(/i)  -  g(jj,))2},  where  g(£i)  is  the  EB  estimate  of  or,  as  Morris 
(1983b)  suggested,  one  can  calculate  the  95%  unconditional  EB  confidence  interval, 
Pa{g(li)  G  C(y)}  >  .95.  Rubin  (1984)  argues  it  is  preferable  to  have  probability 
statement  conditional  on  an  appropriate  data  summary.  In  that  case  one  would  try 
to  find  out  conditional  mean  square  error,  E[(g(n)  -  g(p,))2\b(y)},  where  b(y)  is  a 
suitable  statistic  and  g(fi)  is  the  same  as  before,  or  equivalently,  calculate  the  95% 
conditional  EB  confidence  interval,  PQ{g(fx)  G  C(y)\b(y)}  >  .95. 

Before  we  discuss  this  conditional  and  unconditional  inference  in  a  EB  framework 
we  define  the  concept  of  sufficiency  and  ancillarity  (see  Hill,  1990)  for  our  EB  model. 
Here  we  define  [U\V]  as  a  generic  notation  for  the  conditional  distribution  of  U  given 
V,  and  [U]  as  the  marginal  distribution  of  U. 

Definition  1.  A  statistic  t  is  sufficient  for  #(//)  in  the  EB  model  if  and  only  if 

(a)  t  is  sufficient  for  g(n)  in  the  posterior  family,  i.e. 
[g(n)\t,a]  ss  [g(n)\y,a],  for  all  a  G  A;  and 

(b)  t  is  sufficient  for  a  in  the  marginal  family,  i.e.  [y\t,  a]  =  [y\t],  for  all  a  G  A. 
Definition  2.  A  statistic  a  is  ancillary  for  g(n)  if  and  only  if  t  =  (a,  a)  is  minimal 
sufficient  for  g(jt),  where  d  is  an  estimate  of  a  and  [a\a]  =  [a],  for  all  a  G  A. 

The  above  definitions  are  illustrated  in  the  following  example  of  the  normal  bal- 
anced EB  model  with  known  sample  variance  a2.  The  model  is  given  below: 

{vM  *~  N(j*j,  °2ln)  ,      ~  Ar(/i,  r2);  (fx,  t2)  G  (-00,  +oo)  x  (0,  +oo),  j  =  1,  . . .  k}, 
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In  this  case  the  EB  sufficient  statistic  (Hill,  1990)  for  fij  is  (yj,  y,  Y.(Vj  -  y)2)  and  the 
EB  ancillary  statistic  is  a,  =  -^=J|=L=.  Thereby,  t  =  (yj,  y,  J2(Uj-y)2)  which  is  one 

to  one  with  (/},  f2,  a,)  =  (a,  a,),  where  fi  =  y  and  f2  =         —  y)2/k. 

As  argued  by  Hill  (1990),  the  conditional  distribution  [fij,  t\  a,j,  a],  which  he  called 
the  conditional  likelihood,  should  be  the  reference  distribution  for  conditional  infer- 
ence in  a  EB  model.  However,  the  first  question  is,  why  should  we  worry  about  con- 
ditional inference  anyway?  There  are  at  least  four  general  and  convincing  arguments 
that  justify  conditioning  in  a  frequentist  model.  First,  Fisher's  original  arguments 
concerned  the  recovery  of  information  in  the  sample  lost  by  using  the  maximum  like- 
lihood estimate  alone.  Second,  Hinkley  (1980)  provided  a  very  persuasive  argument 
based  on  the  idea  that  the  inference  should  be  based  on  models  that  have  passed 
model  checking  criteria.  He  observed  that  the  joint  density  of  the  data,  after  a  mini- 
mal sufficient  statistic  reduction,  is  the  product  of  two  components:  the  conditional 
distribution  of  the  maximum  likelihood  estimator  given  the  ancillary,  p(a\ac,a),  and 
the  distribution  of  the  ancillary,  p(a),  which  does  not  depend  on  the  parameter  a.  He 
also  observed  that  most  model  checking  procedures  and  test  statistics  are  based  on 
ancillary  statistics.  He  argued  that  the  distribution  for  the  ancillary  statistic  can  be 
used  to  verify  or  criticize  the  model  and  inferences  about  a  should  be  conditional  on 
the  validity  of  the  model,  that  is  conditional  on  passing  model  checking  tests.  Third, 
conditional  confidence  intervals  often  satisfy  the  necessary  unconditional  properties 
as  well.  Fourth,  conditional  inferences  often  match  Bayes  inference  with  respect  to 
noninformative  priors  (see  Hill,  1990).  Barndorff-Nielsen  (1980)  wrote  (p.  293), 


A  conditionality  resolution  of  a  statistical  model  p(x,  a) 
is  established  by  determining  a  one-to-one  transforma- 
tion of  the  minimal  sufficient  statistic  to  a  new  statistic 
(d,  a),  where  d  is  the  maximum  likelihood  estimate  of  a 
and  a  is  an  ancillary  statistic,  and  then  finding  a  manage- 
able expression  for  the  conditional  expression  p(a\a,a). 
Such  a  resolution  may  be  exact  or  approximate,  depend- 
ing whether  a  is  exactly  distribution  constant  and  on 
whether  the  expression  for  p(d|a,  a)  is  exact  or  approx- 
imate. 

Hill  (1990),  summarized  the  justification  behind  this  conditioning  in  an  EB  model. 
He  gave  four  reasons  for  using  the  distribution,  [/J,j,t\ a,j,  a]: 

(i)  it  accounts  for  the  inherent  variability  of  /ij  by  averaging  over  p,y, 

(ii)  it  accounts  for  the  variability  due  to  estimation  of  a  by  d  by  averaging  over  d; 

(iii)  it  accounts  for  model  checking  by  conditioning  on  Oj', 

(iv)  it  captures  all  the  information  in  the  minimal  sufficient  statistic  t  =  (d,Oj). 
For  the  above  normal-normal  example  with  a2  known,  the  three  statistics  p,  f  2 

and  a,j  are  independent  and  the  distribution  of  aj  does  not  depend  on  a.  For  this 
model  the  conditional  EB  likelihood  of  //,  and  y  is  given  by 

[fij,t\aj,a]   =   [/ijyJb,  &(<*/,«*] 

=  [tij,Vj\&,aj,ot]-[a\aj,a] 
=  [Mj\Vj,ot]-[a\a] 

The  last  equality  is  due  to  Basu's  well  known  theorem  on  complete  sufficient  statistic 
and  ancillary  statistics. 

For  this  example,  the  EB  estimate  of     is  given  by  fij  =  (1  -  B)yj  +  Bp,,  where 

B  =  f 2+<r"/n 1  According  to  Morris'  definition,  a  95%  EB  confidence  interval,  C(y), 
for  [ij  must  satisfy 

Pa[fj,j  G  C(y)]  >  .95  for  all  a  e  A,  which  we  can  write  equivalently  as 
P°[t*j  ~  H  6  C*(y)}  >  .95   for  all   ot  e  A, 
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where  the  probability  is  calculated  with  respect  to  the  joint  distribution  of  fij  and 
y.  This  is  an  unconditional  EB  property,  so  as  Rubin  (1984)  has  observed,  "this  is 
a  fairly  weak  statement  in  the  absence  of  statements  about  conditional  calibration" 
(p.  1163).  On  the  other  hand  according  to  Hill  (1990)  an  appropriate  95%  confidence 
interval  must  satisfy  Pa[fij  G  C(y)\a,j]  >  .95  for  all  a  6  A,  which  we  can  write 
equivalently  as 

Pa[Hj  -  fij  G  C*{y)\  aj]  >  .95   for  all   a  G  A, 

where  a,-  is  EB  ancillary  for  fij.  So  we  see  that  the  problem  is  to  estimate  the 
distribution  of  [fij  —  fij  \  a]  and  [fij  —  fij  |  aj,  a].  If  these  two  distributions  are  known 
then  one  can  calculate  the  conditional  and  unconditional  mean  square  errors  (or 
standard  errors)  or  the  conditional  and  the  unconditional  confidence  intervals.  The 
conditional  standard  error  is  defined  as 

\jEa[(fij  -  fij)2  |oj], 

where  the  expectation  is  taken  over  the  joint  distribution  [/ij,  y|  aj,  a].  There  are  two 
potential  problems  in  deriving  the  above  conditional  distribution.  The  primary  one 
is  the  existence  of  the  EB  ancillary  statistic.  In  the  discrete  cases  that  we  study  in 
this  dissertation,  and  in  fact  rather  generally,  exact  ancillary  statistics  are  not  avail- 
able. The  second  problem  is  even  if  the  ancillary  statistic  exists  the  distribution  is 
not  always  analytically  tractable.  Hill  (1986)  has  shown  that  if  the  ancillary  statis- 
tic aj  exists,  then  we  can  use  the  parametric  bootstrap  technique  to  estimate  the 
distribution  [/x^,  y\  aj,  a]. 

The  general  idea  behind  a  bootstrap  is  to  create  an  estimate  of  the  probability 
mechanism  that  generated  the  actual  outcome,  which  can  then  be  used  to  simulate 
outcomes  from  replications  of  the  experiments.  So  in  this  case  a  bootstrap  estimate 
of  [fij  —  fij\aj,  a]  is  given  by 

[fi*  -  fi*\aj,a], 
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where  only  a  is  replaced  by  a.  Now  since  this  conditional  distribution  is  also  un- 
known, we  use  Monte  Carlo  simulation  technique  to  estimate  this  distribution.  So  for 
the  normal-normal  problem  discussed  above  we  have 

(a)  simulate  n*  ~  N(y,  (a2/n  +  f2)/k)  and  £(%  -  V?  ~  (*V*  +  ^2)  X*-i> 

(b)  calculate  y*  =  p*  +  aj^EiVj  -  y)2/k, 

(c)  simulate  //*  ~  N{(1  -  B)y*  +  Bfi,  (1  -  B)a2/k), 

(d)  calculate  /t*  =  (1  -  B*)y^  +  B*n*, 

where  = 

(e)  calculate  /i^  - 

One  repeats  steps  (a)-(e)  a  large  number  of  times  to  get  the  Monte  Carlo  estimate 
of  the  distribution  of  [fij  —     |  a,j,  &]. 

It  is  very  important  to  note  that  the  balanced  or  equal  variance  normal  model 
is  special  in  almost  every  way  imaginable;  specifically  basic  results  for  the  balanced 
normal  model  rarely  generalize.  For  example  we  consider  the  model: 

Ujl^j  l~  Normal(nj,  Vj) 
Hj  l~  Normal(0,  r2) 

where  Vj  —  o2/rij,  a2  is  the  sampling  variance  of  each  of  those  observations.  This 
model  is  considered  as  'unbalanced  normal-normal  model'.  The  inferential  version  of 
this  model  is  given  by 

Hj\y  $  Normal((l  -  Bj)yj,  (1  -  Bj)Vj) 

Vj  ~  Normal{0,  Vj  +  r2) 

where  Bj  =  Vj/(Vj  +  r2),  for  j  =  1,  ...,k.  The  posterior  family  for  fij  depends  on 
the  data  y  only  through  yj.  Therefore,  fjj  is  Bayesian  sufficient  for  [ij.  The  above 
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marginal  distribution  for  yj  is  a  curved  exponential  family  (Efron,  1975).  Inference 
about  t2  is  much  more  complicated  than  when  Vj  are  equal.  In  this  case  maximum 
likelihood  estimate  of  r2  cannot  be  written  in  closed  form.  Furthermore,  there  is  no 
exact  ancillary  statistic  based  on  yj. 

Hill  (1990)  noted  that  an  approximate  solution  could  be  based  on  second  order 
conditionality  results  of  Efron  and  Hinkley  (1978)  and  Barndorff-Nielsen  (1980,  1983). 
The  expression  for  the  Efron  and  Hinkley  ancillary  statistic  for  this  EB  model  is 
given  by  Hill  (1986,  p.  101)  and  denoted  by  af  for  j  =  1,  . . . ,  k.  Although,  Hill 
introduced  this  second  order  EB  ancillary  statistic  for  conditional  EB  inference,  he 
did  not  consider  the  distribution  of 

[frj  -  Vj\af\r2}, 

where  fij  is  the  EB  estimate  of  //.  Instead,  Hill  used  standardized  residuals 

m  m  vi 


7  ffi+f2 

as  approximate  ancillary  and  considered  the  inferential  model 
A  bootstrap  estimate  of  [fij  -  fij  |  a)1',  r2]  is  given  by 

where  only  r2  is  replaced  by  f2.  This  distribution  is  also  analytically  intractable  but 
can  be  approximated  by  Monte  Carlo  simulation.  For  each  j,  the  following  bootstrap 
algorithm  was  repeated  a  large  number  of  times: 
(a)  simulate  data  from  [f2*  |  af\f2], 

which  in  this  case  is  a  normal  distribution  with  known  mean  and  variance. 
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(b)  calculate  y*  =  af^Vj  +  f2*, 

(c)  simulate  $  ~  N((l  -  Bj)y*,  (1  -  Bj)Vj), 

|  ■      where  Bj  =  Vj/iVj  +  f2),  |      , I ■  ^ '  '| 

(d)  calculate  f%  s  (1—  £*)y* 

where      =  ^/(^  +  f2*), 

(e)  calculate  //*  — 

We  note  that  since  [f2  |a^,r2]  is  a  known  normal  distribution  in  this  case,  Hill 

used  af^  in  place  of       as  the  conditioning  variable.  So  he  had  used  the  second 

order  ancillary  statistic  af^  together  with  afP  to  find  out  the  desired  conditional 
distribution. 

Thus,  in  EB  models  with  a  normal  sampling  distribution  and  normal  prior,  Hill 
(1990)  preferred  conditional  standard  error  calculation  over  unconditional  and  con- 
sidered exact  or  approximate  ancillary  statistics  as  the  conditioning  variables. 

However,  he  did  not  show  any  result  concerning  what  difference  it  would  make,  if 
any,  by  doing  this  more  complicated  and  more  computationally  intensive  conditional 
analysis. 

We  now  consider  the  EB  model  where  the  data  yj,  for  j i  —  1,  . . . ,  k,  are  assumed 
to  have  independent  distributions  belonging  to  a  one  parameter  natural  exponential 
family  with  quadratic  variance  function  (NEF-QVF),  where  the  means  /i/s  are  in- 
dependently sampled  from  the  conjugate  prior  (CP)  distribution.  We  note  that  this 
model  includes  the  unbalanced  normal-normal  case  but,  in  general  exact  standardized 
residuals  are  not  available. 

Carlin  and  Gelfand  (1991),  proposed  a  different  method  for  the  interval  estimation 
problem  in  this  EB  model.  These  authors  first  formalize  their  objective  in  terms 
of  nominal  conditional  coverage.  Their  approach  is  to  start  with  a  naive  interval 
(computed  on  the  basis  of  estimated  posterior)  and  then  to  calibrate  its  coverage 
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using  the  bootstrap.  They  suggest  conditioning  on  yj  and  do  investigate  the  notion 
of  approximate  ancillarity.  Therefore,  their  integration  is  over 

fcji  Vii «  I  Vj>  o]  =  \Mj  I  Vh  aH<*  I  Vjr  a]. 

Now  we  describe  very  briefly  Carlin  and  Gelfand's  method  of  calibrating  naive  EB 
confidence  interval.  We  write  an  equi-tailed,  95%  naive  confidence  interval  as 

(q.025(yj,a) ,  g.975(yj,a)j 

Then  they  define  the  true  coverage  of  the  lower  naive  interval  as 

r(d,  a,  Vj,  tj)  =  PN  \y.  |  Hj  <  QniVj, ")  j 

and 

One  then  solves 

R(a,yj,ri)  =  Mb 

for  77  =  rj(oc,yj,  .025).  This  equation  is  not  solvable  as  it  stands  since  a  is  unknown. 
However,  a  parametric  bootstrap  estimate  is  obtained  by  substituting  d  in  place  of 
a;  that  is  fj  =  rj(6c;  yj,  .025).  The  bias  corrected  lower  confidence  interval  is  then 

From  the  above  discussion  we  see  that  the  bootstrap  technique  was  used  by  both 
Hill  (1986),  and  Carlin  and  Gelfand  (1991)  to  calculate  standard  errors  of  the  EB 
estimates.  Hill  (1986,  pp  77)  used  1000  resamples  to  approximate  bootstrap  standard 
errors.  Efron  (1987)  addressed  the  question  of  the  number  of  bootstrap  resamples 
required  to  accurately  approximate  a  single  bootstrap  standard  error  (see  also  Efron 
and  Tibshirani,  1993,  section  6.4).  His  rule  of  thumb  says  that  the  number  of  resam- 
ples as  small  as  50  is  good  enough  to  get  a  reasonable  bootstrap  estimate  of  the  true 
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standard  error.  However,  with  50  or  even  with  1000  resamples  the  Monte  Carlo  esti- 
mates of  the  bootstrap  standard  errors  are  not  same  when  the  technique  is  repeatedly 
applied  to  the  same  data.  Thus  one  should  choose  the  number  of  resamples  in  such  a 
way  so  that  the  Monte  Carlo  simulation  error  is  negligible.  Efron's  criterion  fails  to 
accomplish  this  objective. 

1.2    Scope  of  the  Dissertation 

In  Chapter  2  of  this  dissertation,  we  consider  simultaneous  estimation  of  several 
local  area  means,  where  each  local  area  contains  a  finite  number  of  elements.  We 
have  considered  the  super  population  of  the  natural  exponential  family  (NEF)  with 
quadratic  variance  functions  (QVF)  (cf.  Morris  1983b).  We  note  that  binary  models 
constitute  a  subclass  of  this  NEF-QVF  class.  The  main  objective  of  Chapter  2  is  to 
carry  out  an  EB  analysis  for  the  NEF-QVF  superpopulation  with  conjugate  priors, 
and  prove  "asymptotic  optimality"  of  the  EB  estimates  of  the  stratum  means  in  the 
sense  of  Robbins  (1955).  An  estimating  function  approach  is  used  to  estimate  the  prior 
parameters,  often  referred  to  as  hyperparameters.  Since  the  normal  distribution  is  a 
special  case  of  the  NEF-QVF  family,  our  results  include  those  of  Ghosh  and  Meeden 
(1986).  Also,  our  results  provide  the  asymptotic  optimality  of  the  EB  estimators 
in  the  general  NEF-QVF  family,  which  to  our  knowledge  has  never  been  addressed 
before. 

As  we  have  mentioned  before,  one  major  problem  in  EB  analysis  is  to  correctly 
calculate  the  standard  errors  of  EB  estimates.  In  Chapter  3  we  provide  a  methodology 
to  calculate  the  correct  conditional  and  unconditional  standard  errors  of  the  EB 
estimates  using  the  parametric  bootstrap.  These  methods  are  then  used  to  calculate 
the  standard  errors  of  the  EB  estimates  from  the  model  introduced  in  Chapter  2. 
Furthermore,  we  provide  mathematical  results  showing  when  the  conditional  and  the 
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unconditional  distributions  of  the  EB  estimates  are  asymptotically  the  same  and  when 
they  differ. 

In  Chapter  4  we  use  asymptotic  methods  to  develop  rules-of-thumb  for  the  num- 
ber of  resamples  required  to  accurately  approximate  bootstrap  standard  errors  and 
variance-covariance  matrices. 


CHAPTER  2 


EMPIRICAL  BAYES  ESTIMATION  OF  MEANS  WITH  STRATIFIED 
SAMPLES:  SAMPLES  TAKEN  FROM  THE  EXPONENTIAL 
FAMILY  DISTRIBUTIONS 

2.1  Introduction 

It  is  been  noted  before  that  normal  theory  plays  a  key  role  in  much  of  the  existing 
literature  in  finite  population  sampling.  Ghosh  and  Lahiri  (1987)  studied  the  proper- 
ties of  the  EB  estimators  in  natural  exponential  family  models  under  the  assumption 
of  posterior  linearity.  However,  they  assumed  prior  variance  to  be  constant  and  the 
prior  means  to  be  equal  for  all  the  small  areas. 

In  Section  2.2  we  introduce  an  EB  model  that  will  generalize  the  one  by  Ghosh 
and  Lahiri  (1987)  and  consider  simultaneous  estimation  of  several  strata  means  in 
the  presence  of  auxiliary  information.  We  take  an  estimating  function  approach  to 
estimate  the  prior  parameters. 

In  Section  2.3  we  show  how  our  methodology  can  be  applied  to  an  overdispersed 
generalized  linear  model  problem  in  a  small  area  setting.  The  developed  methodology 
is  applied  to  a  data  set  whose  purpose  is  to  examine  the  relationship  between  principal 
life  time  occupation  and  cognitive  impairment  for  an  elderly  cohort  in  France.  In  this 
example  there  are  23  different  occupational  categories  which  can  be  treated  as  the 
different  small  areas.  The  number  of  samples  in  those  small  areas  are  sometime 
very  small;  for  example,  for  the  last  category  it  is  equal  to  20.  Therefore,  ordinary 
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proportion  estimates  for  the  categories  with  small  sample  sizes  may  be  subject  to  large 
standard  errors,  and  one  needs  to  borrow  strength  from  areas  with  larger  sample  sizes 
to  obtain  better  estimates  of  the  true  probabilities. 

Ghosh  and  Lahiri  (1987)  studied  the  asymptotic  behavior  of  their  EB  estimates. 
Similar  "asymptotic  optimality"  results  for  our  EB  estimates  are  given  in  Section  2.4 
of  this  chapter. 

Section  2.5  contains  proofs  of  some  of  the  more  technical  results. 


Let  Yji  denote  the  characteristic  of  interest  with  the  i  th  unit  in  the  j  th  stratum 

(i=  1,  ,Nj-,  j=  1,  ,  k).  A  fixed  sample  size  rij  (<  Nj  '—  1)  is  taken  from  the 

j  th  stratum.  Without  loss  of  generality,  these  sample  values  are  denoted  by 


Our  problem  is  to  estimate  77  =  ^-  Y.d\  Yji  for  j  =  1,  ,  k.  under  squared  error 

loss.  As  pointed  by  Ghosh  and  Meeden  (1986),  this  problem  can  be  identified  as  a 
classical  prediction  problem.  The  model  we  are  going  to  assume  is  as  follows: 


2.2    The  Estimation  Procedure 


yj  =  (yju  ■■■■,yjnj)T;  v3- 


Vii\d3  ~  eXP\<t>3  {VjiOj  -  Wi)}}  X  C(%i,0j), 


(2.2.1) 


for  i  -  1,  ,n,j  and  j  =  1,  ...,k.  Under  the  given  model 


E(yji\8J)  =  b'(6j)  =  nj 


and 


Var(yji\d3)  =  fftOM  =  Vtmy4>i  (say). 


For  the  NEF-QVF  family, 


VM  =  V0+  Viflj  +  V2IXj2. 
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Symbolically  we  shall  write 

Vjt\  Oj  ~  NEF  -  QVF  (jMj,  VM/h),    for  all  i,  j. 
This  further  implies  that 

-  M*J  -  QVF     K#i)/n,fc);     i  =  1,  •  •  -  fc.  (2.2.2) 
Consider  the  conjugate  prior 

6j  ~rf  ezp[  A  {mjdj  -  b(6j)  } }  x  JC(A,  m/j;     j  =  1,  . . . .,  k,  (2.2.3) 

where  E(jij)  —  rrij,  is  the  prior  mean  for  the  jth  local  area.  The  posterior  pdf  of  6j 
is  then 

7r(0j  I  jjj)  oc  exp  [  6j  (rij  <f>j  yj  +  A  rrij)  -  (nj  <f>j  +  A  b(6j) )  ].  (2.2.4) 
Write  Bj  =  A/(A  +  rij(pj).  Then  the  posterior  mean  of  /ij  =  6  (0,)  is  given  by 

BitH  I      =  (1  -         +  B7mr  (2-2-5) 

The  Bayes  estimator  of  7  =  (71,  . .  .,^k)T  is  then  given  by  (cf.  Ericson  1969,  Ghosh 
and  Meeden  1986)  eB  =  (elB,  . .  .,ekB)T,  where  eB  =  jjj  -  fjBffy  -  rrij), 

fj  =  (Nj-nj)/Nj,  (j  =  l,....,k). 
We  model  the  rrij  as 

g(mj)  =  xj(3,  {j  =  l,....,k),  (2.2.6) 

where  g(-),  referred  to  as  the  link  function,  is  monotonically  increasing  in  rrij.  This 
Bayesian  model  is  similar  to  Albert  (1988).  Our  model  generalizes  the  one  of  Ghosh 
and  Lahiri  (1987)  in  several  respects.  They  assumed  that  the  mean  parameters 
Hi,  ...,Hk  are  iid  with  mean  //  and  variance  a2.  This  is  the  setting  of  a  random 
effects  one  way  ANOVA  model  and  does  not  cover  the  regression  case.  Secondly,  we 
are  not  restricted  to  the  identity  link  function.  We  also  note  that,  unlike  Ghosh  and 
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Lahiri  (1987),  we  shall  exploit  the  fact  that  the  sampling  variance  is  a  known  function 
of  the  sample  mean  in  the  subsequent  EB  analysis. 

Our  Bayesian  model  can  be  viewed  also  as  a  mixture  model  and  is  particularly 
suitable  to  account  for  overdispersion.  The  marginal  mean  and  variance  are  given  by 

Efo)  =  =  E9i{i*j\  =  mj 

and 

Varivj)  =  (^njy'EiVifXj)}  +  Var{H) 


respectively.  Thus,  even  if  rij  — >  oo,  Var(yj)  — >  A(A  -  V2)~lV(m,j)  ^  0.  This  is 
one  criticism  of  the  mixture  model  as  voiced  in  Gelfand  and  Dalai  (1990).  Such  a 
criticism,  however,  seems  of  little  relevance  in  the  small  area  context  since  the  rij 
are  typically  small.  We  may  also  note  that  when  A  — >•  oo,  the  overdispersed  model 
essentially  reduces  to  the  NEF-QVF  model  in  (2.2.2)  with  rrij  taking  the  role  of  fij. 
This  is  equivalent  to  the  selection  of  a  point  prior.  Thus,  for  infinitely  large  A  our 
model  converges  to  a  generalized  linear  model  (McCullagh  and  Nelder,  p.  28,  1989). 

In  order  to  apply  EB  we  first  require  an  estimation  procedure  for  (3  and  A.  Initially 
we  assume  that  A  is  known.  Then  to  estimate  /3,  we  use  the  optimal  estimating 
equations  from  the  marginal  distribution  of  the  sample  observations.  The  set  of 
equations  is  given  by 

*  dmj       X-v2       nrf^  -  mj)  _  xl 

£  d0    a  +  4>jnj  x    v(mj)    -  °^  •  (2-27) 

The  estimating  equation  (2.2.7)  is  optimal  in  the  sense  that,  for  fixed  A,  it  has  the 
minimum  asymptotic  variance  in  the  class  of  all  linear  unbiased  estimating  equations. 

Since       "  V^Wy  writin§  %  =  wiW  =  ^fgf .  (2-2-7)  can  be  rewritten  as 

Sh0)  =  k~l  g  Wj  x       {V]~^X\^  =  0**1  (2.2.8) 
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For  the  special  case  of  a  canonical  link  g'(rrij(($))  —  V~l(mj((3))  and  (2.2.8)  simplifies 
to  Wj  (yj  —  rrij)Xj  =  0.  As  g(-)  is  a  monotone  function,  nij  can  be  written  as 
g~1(xj(3)  =  h(xj(3).  The  estimates  obtained  from  (2.2.8)  are  known  as  maximum 
quasilikelihood  estimates  as  defined  by  Wedderburn  (1974).  In  fact  for  fixed  A,  the 
marginal  quasilikelihood  for  (3  based  on  the  sample  has  the  same  form  as  the  NEF 
sampling  likelihood  for  fij,  with  known  but  different  weights.  Moreover  as  A  — >•  oo  in 
(2.2.8),  Wj  — >  rij(t)j.  This  implies  that  for  sufficiently  large  A,  the  estimator  of  (3  in 
(2.2.8)  will  reduce  to  ordinary  maximum  likelihood  estimator  under  the  regular  one 
parameter  exponential  family  model. 

We  now  consider  the  more  realistic  situation  where  both  (3  and  A  are  unknown. 
We  write  (/3,  A)T  =  0fr+1>xl. 

We  estimate  A  by  the  method  of  moments,  and  use  the  between  block  sum  of 
squares  for  this  purpose.  Let  y  =  Y.njVjl  Y,nj  and  fh  =  Y.^j'^il 'J2nj-  We  use  the 
identity 


E 


k 

E 


(\  —  ^  ^WjV(rrij)  -f  rij(mj  —  m)' 


to  obtain  the  estimating  equation 


1 


where 


UjiO) 


5>,(0)  =  O, 


(2.2.9) 


N  V(mj(P))g?{mi{P)) 

K  njifj  -  yf  gj^jwi  V{mj)  -  n,(m,-  -  fh)2 


(2.2.10) 


So  the  estimating  equations  for  estimating  6  e  0  C  il(p+1)  are  written  as 


Sk{0)  =  k-lYiuj{0)=0. 


(2.2.11) 


19 


For  unknown  (3  and  A  our  EB  estimator  of  7  is  denoted  by  eEB,  where 

eEB  =  yj-  fjBjiVj  -  fhj), 

for  j  =  1,  . . .  ,k.  Here  rhj  is  the  estimated  prior  mean  for  jth  local  area,  the  details 
of  the  estimation  procedure  is  given  in  Section  2.4.  For  N*  =  00,  fj  =  1  and  in  that 
case  we  denote  e?EB  by  ftj,  where 

H  =  (!  -  Bj)Vi  +  Bi™v 

the  estimated  posterior  mean.  In  the  next  section  we  will  see  how  the  above  method- 
ology can  be  applied  to  a  real  life  problem. 

2.3    An  Application 

Dartigues  et  al.  (1992)  examined  the  relation  between  principal  lifetime  occupa- 
tion and  cognitive  performance  in  a  cohort  of  3,777  community  residents  in  France. 
Subjects  were  considered  cognitively  impaired  if  they  scored  under  24  on  the  mini 
mental  state  examination  (MMSE).  Distribution  of  the  23  occupational  categories 
and  proportion  of  subjects  with  a  MMSE  score  lower  than  24  are  presented  in  Table 
2.1.  In  column  one  we  have  the  name  of  the  principal  life  time  occupation.  In  column 
two  we  have  the  number  of  people  sampled  (rij)  in  those  lifetime  occupational  cate- 
gories and  in  column  three  we  have  the  actual  proportions  diagnosed  as  cognitively 
impaired.  We  denote  as  the  sample  proportion  from  the  jth  occupational  category. 
Here  j  runs  from  1  to  A;  with  k  equal  to  23.  First  we  note  that  the  sample  sizes  vary 
considerably  over  different  occupational  categories.  The  maximum  sample  size  is  383 
for  farm  managers,  while  the  minimum  sample  size  is  20  for  inactives.  Moreover,  the 
proportions  vary  considerably  from  0  percent  in  qualified  school  teachers  and  profes- 
sor to  54.1%  in  farm  workers.  Evidently,  these  sample  proportions  are  not  always 
reliable,  especially  for  the  categories  with  small  samples.  For  instance,  a  zero  could 
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be  just  because  of  a  very  small  sample  size  in  that  category.  In  order  to  fit  a  model, 
one  can  assume 

niVj  '~  bin(nj,       for  j  =  1,  ...,k 

and  model 

logit{^)  =  p. 

This  is  the  logistic  regression  model  with  only  the  intercept  parameter  p.  However, 
here  this  model  does  not  fit  well  (x2=213.2  with  22  df).  As  discussed  in  Section 
2.2,  the  reason  behind  this  lack  of  fit  is  the  variability  in  the  data  not  explained  by 
an  ordinary  binomial  model.  Lack  of  explanatory  variables  in  the  logistic  regression 
model  could  be  the  cause  of  overdispersion  here.  Also  the  sample  proportions  are 
estimates  from  a  saturated  logistic  regression  model 

logit^j)  =  p j. 

This  model  has  a  x2  value  of  zero  with  zero  degrees  of  freedom.  These  logistic 
regressions  represent  two  extremes.  In  the  first  case  we  have  an  exchangeable  model 
with  one  parameter  for  all  the  means,  and  in  the  second  case  we  have  a  saturated 
model  where  each  mean  is  modeled  with  a  separate  parameter. 

In  order  to  fit  our  empirical  Bayes  model  we  first  assume  conjugate  prior  distri- 
bution on  the  mean  parameter  pj.  The  mixture  model  is  more  parsimonious  than 
the  saturated  one  because  it  uses  a  parametric  distribution  to  describe  variability  in 
the  intercept  terms.  Here  with  tijjfo  following  binomial,  the  conjugate  prior  is  a  beta 
distribution.  Hence, 

Hj  <v  beta(m\,  (1  -  m)A)  for  j  =  1,  . .  .k, 
and  model  the  marginal  mean  m  as 

logit(m)  =  p. 


21 


Here  we  assume  both  0  and  A  are  unknown  and  estimate  them  by  using  the  estimating 
equation  technique  discussed  in  Section  2.2.  The  estimates  are  m  =  0.195,  and 
A  =  7.313.  The  EB  estimates  of  the  means  are  given  in  column  4  of  Table  2.1. 


Table  2.1:  The  EB  estimates  of  the  means  from  the  cognitive  impairment  data 
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Private  middle  executives 

187 

.075 

.0795 

.0193 

.0194 

11 

Private  white-collar  workers 

181 

.091 

.0950 

.0214 

.0213 

12 

Business  employees 

136 

.135 

.1381 

.0293 

.0287 

13 

Private  executives,  managers 

135 

.061 

.0679 

.0206 

.0210 

14 

Policemen,  soldiers 

98 

.062 

.0712 

.0244 

.0249 

15 

Primary  school  teachers 

68 

.060 

.0731 

.0288 

.0298 

16 

Public  executives 

61 

.068 

.0816 

.0322 

.0329 

17 

Drivers 

60 

.241 

.2360 

.0552 

.0514 

18 

Public  middle  executives 

55 

.017 

.0379 

.0174 

.0239 

19 

Teachers,  lecturers 

48 

.000 

.0258 

.0000 

.0211 

20 

Nurses 

43 

.171 

.1745 

.0574 

.0529 

21 

Professionals 

30 

.138 

.1492 

.0629 

.0576 

22 

Other  occupations 

25 

.080 

.1061 

.0543 

.0534 

23 

Inactives 

20 

.450 

.3818 

.1112 

.0913 

Here  we  have  considered  Nj  =  oo.  We  note  that,  if  sample  size  (n,)  is  large  then  the 
EB  estimates  are  very  close  to  ordinary  proportions  (estimates  from  the  saturated 
lrThe  naive  s.e.  for  /i,  is  ./S^M. 

™       Y  A+ny+1 
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model).  Otherwise,  they  tend  to  shrink  towards  the  overall  mean,  which  in  this 
case  is  0.24  (estimate  from  the  exchangeable  model).  Therefore,  our  empirical  Bayes 
estimates  serve  as  a  compromise  between  the  two  estimates  obtained  before.  In  a 
EB  approach,  one  treats  (m,  A),  as  fixed  unknowns  and  replaces  them  with  (m,  A)  in 
the  posterior  means  and  variances.  In  Theorem  5  of  the  next  section  we  demonstrate 
that,  as  an  estimator  of  fij,  the  EB  estimate  (fij)  given  in  column  four  performs  well 
in  a  decision  theoretic  sense.  Unfortunately,  naive  standard  errors,  calculated  from 
the  estimated  posterior  variances,  are  generally  too  small.  The  explanation  for  this 
problem  from  a  EB  point  of  view  is  that  we  are  ignoring  the  variability  in  (m,  A).  For 
detailed  discussion  of  this  problem  we  refer  to  the  next  chapter  of  this  dissertation. 

In  this  case  the  expression  of  the  naive  standard  error  for  fij  is  and  is  given 

in  the  last  column  of  Table  2.1. 

By  borrowing  strength  one  would  expect  to  estimate  the  means  more  precisely.  In 
other  words,  the  naive  standard  errors  should  be  smaller  than  the  ordinary  standard 
errors  given  in  column  six.  But  we  can  easily  see  why  this  is  not  always  the  case.  To 
get  a  smaller  naive  standard  error  for  the  j  th  mean  fij,  we  should  have 

-  Aj)  <  x    A  +  i 
Vj^-Vj)  ~  rij 

which  is  always  true  if  fij  <  <  0.5.  This  is  the  case  for  most  of  the  categories  in 
Table  2.1.  But  if  yj  <  fij  then  the  naive  standard  error  will  be  larger  than  the  sample 

standard  error  if  (A  +  l)/rij  is  very  small.  This  is  the  case  for  categories  14-16  and 
18-19. 

We  have  noted  that  the  naive  standard  errors  are  typically  too  small.  Thereby, 
calculation  of  corrected  standard  errors  are  important.  One  could  be  interested  in 
computing  conditional  and  unconditional  mean  square  errors,  E[{fij  -  fij}2\y~j)  and 
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E[{fij  -  fij}2},  where  fij  is  the  EB  estimate  of  j  th  mean  fij.  Now  since  these  quan- 
tities are  intractable,  we  estimate  these  quantities  by  using  parametric  bootstrap 
methodology.  The  general  idea  behind  a  bootstrap  is  to  create  an  estimate  of  the 
probability  mechanism  that  generated  the  actual  outcome,  which  can  then  be  used  to 
simulate  outcomes  from  replications  of  the  experiments.  So  in  this  case  a  bootstrap 
estimate  of  E[{fij  -  P>j}2\m,  A]  is  given  by  E[{^  -  #-}2|m,  A].  A  bootstrap  estimate 
of  E[{nj  -  fij}2\yj,  m,  A]  is  given  by  E[{fi*  -  p,*}2\yj,  m,  A]  where  in  both  cases  (m,  A) 

is  replaced  by  known  (ra,  A).  Now  since  these  expectations  also  have  no  closed  form 
expression,  we  use  Monte  Carlo  simulation  techniques  to  approximate  them.  A  de- 
tailed discussion  of  the  conditional  and  unconditional  mean  square  errors  is  given  in 
the  next  chapter. 

In  the  next  section  we  prove  results  related  to  asymptotic  properties  of  the  EB 
estimators  discussed  in  Section  2.2. 

2.4    Properties  of  the  estimators 

At  first  we  consider  the  case  of  known  or  fixed  A.  In  the  first  two  theorems  we 
show  the  consistency  and  asymptotic  normality  of  the  solution  0k  from  (2.2.8).  We 
follow  the  approach  of  Foutz  (1977).  To  this  end,  first  write  the  matrix  derivative 

— Sk($)  =  S'k((3)  =  -\Uk{(3)  +  Rk((3)},  (2.4.12) 

where 
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and 


<?"K(/3)) 


+ 


3  3 


Notice  that,  since  |  /3)  =  mj{/3),  E(Rk{/3))  =  0  and  hence  E(S'k((3))  =  -Uk(0). 
Let  /30  be  the  true  value  of  /3.  The  following  assumptions  are  needed  for  proving  the 
first  result  of  this  section. 

(A.l)  Sk{(30)  ^  0,    as  k  ->  oo 

(B.l)  lim^oo  Uk((30)  =  U((30);  U{0)  is  positive  definite  for  all  0  e  Bs 
=  {(3  :  ||/3  —  /30||  <  J}  and  is  continuous  at  (30 


(C.l)  sup,6Bi  \\S'k((3)  +    (0)||  4  0  as  k  ->  oo;  where  fl,  =  {/3  :  ||/3  -  /30||  <  5}. 


Later,  we  will  provide  sufficient  conditions  in  terms  of  the  design  vectors  and  link 
functions  for  (A.l)  -  (C.l)  to  hold. 

Theorem  1:  Under  the  assumptions  (A.l)-(C.l),  there  exists  a  sequence  {/3k}  of  so- 
lutions to  (2.2.8),  such  that 

0k      (30      as   k  — >  oo 
Further,  if  {0k}  is  another  sequence  of  solutions  to  (2.2.8),  then 

as  k  — >  oo. 

The  proof  of  this  theorem  is  deferred  to  Section  2.5.  The  following  important 
corollaries  can  be  obtained  from  this  theorem. 
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Corollary  1:  Assume  the  conditions  of  Theorem  1.  If 
suPi<j<jfc{[0'(™;(A>))]-1  x  INI}  <  °°> then 


sup  \rhj  —  m,j\  — £  0  as  k  — »•  oo 
i<j<fe 


Proof: 


sup  Im^  -  mj|  =  sup  |/i(x^fc)  -  /i(jcJ/30)| 
i<?<*  i<j<* 


=     sup  {(&  -  /30)T  x  «M«JA)  +  o(||i9*  -  /30l|2)} 
l<j<fc  I  op  ) 


<    \\0k-(3o\\x  sup  {[g^mjifa))}-1  x  \\Xj\\}  +  o(\\Pk  -  (30\\2) 

l<j<k 

Pg 

-4   0  as  k  — ^  oo 

The  condition  of  Corollary  1  holds  in  the  binary  case  if  sup1<J<fc  \  \xj\\  <  oo  since  then 
[SWA)))]"1  =  V(mj(0))  =  mJ(/30)(l  -  mj(0o))  <  1/4. 

The  next  theorem  proves  the  asymptotic  normality  of  the  estimators  of  f3. 
Theorem  2:  Suppose  that  the  conditions  of  Theorem  1  hold,  and  let  {0k}  be  a  consis- 
tent sequence  of  estimators  of  (3  which  satisfy  equations  (2.2.8).  Suppose  in  addition 

where  Wj  are  defined  in  (2.2.8).  Then 

kHp-(30)L-%  MVNp(0,  U((30)-1) 
Once  again  with  canonical  link  the  condition  simplifies  to 
k 

*    £      *  (Vi  ~  «h(fio»*j  4  MVNp(0,  C/^o)-1), 

3=1 
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Next  we  find  out  the  Bayes  risk  of  the  empirical  Bayes  estimator  eEB  of  7,  where 

CEB  =  (eEB,  eEB>  ■  ■  ■  1  eEE))T-  In  tnis  Case 

eEB  =  Vj  -  fjBj(Vj  -rhj), 

for  j  =  1,  . . . ,  fc.  We  denote  by  r(£,  e)  the  Bayes  risk  of  an  estimator  e  of  7  where 
the  prior  £  is  described  in  (2.2.3).  Ghosh  and  Meeden  (1986)  proved  that  for  any 
estimator  e, 

r(C,  c)  -  r(C,  e*)  =  £  £  £(e»"  -  eJB)2,  (2.4.13) 
*  3=1 

where  £'(•)  denotes  expectation  over  the  prior  distribution  C  as  well  as  the  sample. 
Replacing  e  by  eEB  the  right  hand  side  of  (2.4.13)  becomes 

The  first  major  theorem  of  this  section  is  as  follows. 

Theorem  3:  Suppose  in  addition, 

(a)  Bupj^jt  \rhj ;  -  rrij\  A  0  as  k  00 

(b)  E(^j  -  n»i)2)a  <  M,  for  some  positive  number  M, 
where  g(rhj)  =  xj(3,  (3  been  estimated  from  (2.2.8).  Then 

r(C,  £eb)  -  r(C,  eB)  -*  0  as   ft  ->  00. 

Following  Robbins  (1955),  this  property  is  called  asymptotic  optimality  of  the  empir- 
ical Bayes  estimators. 
Proof: 

1  * 

I  E        (fy  ~  mj)2  <  sup  \rhi  -  m,|2  4  0 
K  j=l  l<)<k 
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due  to  (a).  Since  (b)  implies  the  uniform  integrability  of  the  sequence 

{k~1j:B]f^mJ-mj)\k>l}, 

the  result  follows. 

Condition  (a)  holds  under  the  conditions  given  in  Corollary  1.  Condition  (b) 
holds  automatically  for  the  binomial  model  with  canonical  link  since  then  rhj  — 
exp(xj(3)/(l  +  exp(xjfi))  G  [0, 1],  uniformly  in  j  >  1.  As  mentioned  in  the  Section 
2.1,  this  is  the  case  which  has  been  addressed  most  often  in  the  literature  and  Theorem 
3  establishes  the  asymptotic  optimality  property  of  the  EB  estimator  in  this  case. 

For  all  other  cases,  in  order  to  prove  (b)  we  assume  the  canonical  link  and  a 
specific  structure  of  the  design  matrix  Xhxp.  Without  loss  of  generality  we  partition 
{l,...,k)  as  (1, . . .,  ki),  (fci  +  1, . . . ,  k2),  •  •  •  • ,  (A;p.-i  +  1,  •  •  • ,  kp);  where  kp  =  k.  For 
j  =  ks_i  +  X,  . . . ,  k,  it  is  assumed  that  the  ks_i  +  1,  . . . ,  ks  elements  of  the  j  th 
column  vector  of  the  matrix  X  are  1,  and  rest  are  zeros.  As  before,  we  denote  xj 
to  be  the  j  th  row  vector  of  X.  Thus,  for  j  =  1, . . . ,  k\,  with  g  being  the  canonical 
link,  mj  =  g'Hxjp)  =  g~\Pi)  =  m1  (say).  In  general,  for  ;  m  ks_i  +  1,  ...,ks 
™>j  =  g~l{x^(3)  =  g~l(Ps)  =  ms  (say),  where  s  -  1,  . . .  ,p.  Thus  without  loss  of 
generality  we  can  write 

mj  =  m2;     j  =  fcj  +  1,  . . . ,  k2 
mj  =  mp;      j  -  kp-i  +  1,  ...,k 

For  j  =  ks  _i,  . . . ,  ks,  estimate  of  mj  is  equal  to  ms  where 


(2.4.14) 
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Suppose  now  {supfc>1  k  1        E(jft)\  <  oo.  Then 


1  * 


iff  <  ^rE^-^)4] 


<   8x£[-j:(mJ  +  mJ)] 


(  as  rrij  =  E(rhj)  for  all  j  ) 


=  i6x^ 


<  oo 


(  as  2  <  rij^j  <  C  ) 

£  40 


(2.4.15) 


So  condition  (b)  of  Theorem  3  holds  if  the  fourth  moment  of  the  marginal  distribution 
exists. 

Now  we  consider  the  case  where  both  ft  and  A  are  unknown.  We  write  ((3,  \)T  = 

0(P+l)xl_ 

As  we  have  seen  in  Section  2.2,  we  estimate  A  by  the  method  of  moments  and 
use  the  between  block  sum  of  squares  for  this  purpose.  Let  y  =  En^/J^n,-  and 
»™  =  T,njmj/T,nj-  We  use  the  identity 


E 


£  ni  (Pt  -  v)2 

i=i 


k 

t 


71  j 


to  obtain  the  estimating  equation  for  A. 
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Therefore,  the  estimating  equations  for  estimating  0  G  0  C  i^p+1)  are  as  given 
in  (2.2.11).  As  before,  let 


=  £(Sj;(0))  +  Jfe(0) 

=   -{Uk(0)  +  Rk(0)}  (2.4.16) 


where  = 


and 


t»ll(a)  uf(e) 


(say), 


Rk{0) 


{Rk{P)YxP  (Fxl 
0ixp  o 


(2.4.17) 


(2.4.18) 


Further,  we  write,  Wk(0)  =  k~l  Var(Zkj=1Uj(e)),  where  the  variances  and  the  co- 
variances  are  taken  over  the  marginal  distribution.  Thus, 
Wk(G)  = 


E -=1  n?  Var(y-j  -  y) 


r.\2 


(2.4.19) 
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As  before  we  denote  Oq  as  the  true  value  of  0.  To  prove  a  theorem  similar  to  Theorem 
1,  we  further  assume 

(A.2)  0  <  lim^/c"1  Y,n)Var(yj-y)2  <  oo; 

(B.2)  lim^oo ull(90)  —  u21(60),    lim^oo ul2(00)  =  u22(00),  and  are  continuous  at 
0O.  Further  u2\9)  is  finite  and  u22(0)  >  0  for  all  6  e  Bs  = 
{0  :  \\9  -  6o\\  <  6}; 


(C.2)  sup5eB6 


u2\o)-u?(e0) 

uf{9)  -  u22(0o) 


0  as  k  ->  oo  for  all  9  G  Bs- 


Theorem  4:  Under  the  assumptions  (A.l)-(C.l),  and  (A.2)-(C.2)  there  exists  a  se- 
quence {0k}  of  solutions  to  (2.2.11),  such  that 

Ok  — £  6o     as  k  — ^  oo 
Further,  if  {Ok}  is  another  sequence  of  solutions  to  (2.2.11),  then 

Pe0{Ok  ^Ok)^0     as  k  -*  oo 

The  proof  of  the  theorem  is  given  in  Section  2.5.  The  following  corollaries  are 
simple  consequences  of  Theorem  4. 
Corollary  2:  Under  the  assumptions  of  Theorem  4, 

sup  \Bj  -Bj\Ao 

\<j<k 

Proof: 

sup  \Bj  -  Bj\  <  (  sup  rij)  x  | A-1  -  A"1 1  4  0  as  k  ->•  oo. 
i<j<fc  i<i<fc 

Corollary  3:  Under  the  assumptions  of  Theorem  4,  and  under  the  model  given  in 
(2.4.14), 

sup  \wj         4o  as  k  -4  oo 
i<j<k 
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and 


sup 

l<s<p 


«<s  j=ka_1  +  l  fts       K»-l  j=ks-1  +  l 


0  as  (ks  —  ks-i)  — >•  oo 


where  i&*  =s  (A  S2i^j  _ 

J  \+nj<f>j 

Proof:  Using  the  Taylor  expansion 


i  -  it      » i  / nj<l>i  +  ^2 

sup  \Wj  —  Wj\  <  I A  —  A|  x  sup  I  -J-^L 
i<j<k  j   \  ni 


ij(f>j  J 


— >  0  as  k  — ►  oo. 
Proof  of  the  second  part  also  follows  in  a  similar  way. 

Corollary  4:  Under  the  assumptions  of  Theorem  4,  Corollary  1  and  the  model  given 
in  (2.4.14) 

sup  \rhj  —  Trij  \  — >  0 

l<j<k 

where  rhj  =  m,j(0). 
Proof:  Write 


sup  \rhj  -  rrij\  <  sup  \ms  -  fhs\  +  sup  \rhj  -  rrij\,  (2.4.20) 
i<j<fc  i<«<p  i<i<* 


where  rhj  =  mj(/3\)  where  p\  is  the  estimate  of  (5  from  equation  (2.2.8)  with  A  fixed 
or  known.  In  Corollary  1  we  have  already  proved  that 


sup  \rhj  —  mA  — >■  0 
i<j<fc 


With  the  partial  exchangeable  model  (2.4.14), 


sup  \rrij  —  rrij\    =  sup 

1<«<P  1<«<P 


Wi 
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<  sup 

1<4<P 


(ks    ks-i)  1  Yl'jLk.-i 


+1Wj 


x  sup 

l<s<p 


(ks  -  k,-i)  1  yj 


X    SUp   \Wj  —  Wj\ 
l<j<h 


+  sup 

l<s<p 


x  sup 

l<s<p 


(ks  -  k,-i)  1  wiVi 


1 


=  Op(l)  x  Op(l)  x  op(l)  +  Op(l)  x  op(l) 
=  op(l), 

using  Corollaries  3  and  4  and  the  fact  that  both  Wj  and  %  are  random  variables  with 
non  zero  means. 

Now  we  state  and  prove  the  final  theorem  of  this  section.  This  theorem  generalizes 
Theorem  3.  In  this  case  e?EB  =  ffo  -  fjBj(y~j  —  rhj),  and 

1  k 

H<>  eEe)  -  rfti  eB)  =  r  E  fj2  Wft  ~  Bj){yj  -  tUj)  -  Bj{*H  -  ™;)]2, 
for  j  =  1,  . . .  ,k. 

Theorem  5:  Under  the  assumptions  of  Theorem  4,  with  the  model  in  (2.4.14)  it  can 
be  proved  that 

r((,eEB)  -  r(£,eB) ->  0  as   k  oo 

In  addition  we  need  to  assume  sup^  E(y*)  <  oo  and  sup^j  k~x      E(jj)  <  oo. 

Proof:  In  order  to  prove  this,  at  first  we  need  to  prove  that 

|  Ekj=i  fj2  [{Bj  -  Bj){yj  -  rrij)  -  Bj{rhj  -  rrij)]2  4  0  as  k  ->•  oo. 
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Now,  using  the  Corollaries  2  and  4,  we  get 


xrEfe-mi)2l 
*  i=i 


+    2  supi<j<k(rhj  -  m,)5 

4  o, 


as  /c  — v  oo.  Now,  we  need  to  prove  the  uniform  integrability  of 


E  fi  [(4  -  Bj)(yj  ~mi)~  Bjirhj  -  rrij)]2. 


3=1 


To  this  end,  we  use  the  inequality 


k  k 
Lj=l  j=l 


To  prove  uniform  integrability,  it  suffices  to  prove  that  Y.{Uj  ~  mj)2]2  <  oo  and 
E{\  Z{rhj  -  rrij)2}2  <  oo. 

The  first  condition  is  true  if  supjM  E(yjY  <  oo.  To  prove  the  second,  use  the 
inequality 


E[W{m3-m3f]2   <  WE^-mtf 


k 

<   lEEim.-m^  +  ^Elm.-mjY 

(  by  (2.4.15) ), 
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(  where  Wj  =  Wj/  7! Wf-  <  1 ) 
<   00.  (2.4.21) 

The  inequality  (2.4.21)  is  true  by  the  assumption  of  the  theorem. 

2.5  Proofs 

Proof  of  Theorem  1 :  By  assumption  (B.l),  we  define  r  =  1/(4||  C/-1(/30)||)  and  rk  — 
l/(4||{5'fc(/30)}-1||)  and  select  8  €  Bs  sufficiently  small  so  that  (C.l)  holds  and 

\\U(P)-U(0o)\\<t/2.  (2.5.22) 

This  is  because  in  (B.l)  we  assumed  U((3)  is  continuous  at  80. 

Using  (C.l)  we  get  rk  -»  r  in  probability.  Hence  for  /?  G  Bs  =  {(3  :  \\/3-(30\\  <  8}, 

\\S'k((3)  -  S'k((30)\\  <  mfafVim  (2.5.23) 
+  \\S'k((30)  +  U((30)\\  (2.5.24) 
+   \\U((3)-U((30)\\  (2.5.25) 

T       T  T 

<  4+4+2 

<  r  <  2rk.  (2.5.26) 

By  (C.l)  with  probability  ->  1,  (2.4.23)  and  (2.4.24)  can  be  made  less  than  r/4, 
and  by  (2.5.22),  (2.4.25)  is  <  r/2.  Therefore,  (2.5.26)  holds  with  probability  -»  1  as 
k  — >  00. 

Now,  using  the  Inverse  Function  Theorem  (Rudin,  1976),  with  probability  1, 
Sk  is  a  one-to-one  function  from  Bs  to  Sk(Bs)  and  S,fc(B(5)  contains  the  open  circle 
of  radius  rk8/2  about  Sk((30).  Also,  by  (A.l),  0  <E  Sk(B5)  with  probability  ->  1 
as  k  ->  00.  Consider  the  inverse  function  S^1  :  Sk(Bs)  -4  5,$.  It  is  well  defined 
whenever  Sk  is  one-to-one,  that  is  with  probability  ->•  1.  Since  0  G  5*(fitf)  with 
probability  ->  1  as  k  -»  00,  we  conclude  that: 


35 


1.  the  root,       (0),  of  (2.2.8)  exists  in  Bs  with  probability      1  as  A;  -»  oo; 

S.  since  5  may  be  taken  arbitrarily  small,  S^l(0)  converges  in  probability  to  /30; 

3.  Since  Sk  and  5,5  are  one-to-one,  any  other  sequence  {0k}  of  roots  of  (2.2.8)  nec- 
essarily lies  outside  B$  with  probability  — >  1,  that  is  the  sequence  does  not 
converge  to  (30. 

The  proof  is  now  complete  with  j3k  —  S^"  *(()). 

Now  we  find  out  the  conditions  under  which  (A.l)  and  (C.l)  hold.  A  direct 
application  of  Chebyshev  Inequality  will  enable  us  to  show  that  (A.l)  holds  if 


1  k 

x  ^K(/30))^K(/30)X  =  0 


(2.5.27) 


for  all  s  =  1,  . . .  ,p.  Equivalently  this  is  written  as: 


1 


(2.5.28) 


where  Gj  = 


{var^Mm^A)))}  *j 

We  note  that  J$fc(/90)  is  again  an  average  of  A;  independent  centered  random  vari- 
ables with  finite  variances.  Therefore,  Rk(/30)  -¥  0pxp     in  probability  as  k  -»  oo 


if 


1 


(2.5.29) 


i=i 


where  G*  = 


-  x 
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To  prove  (C.l)  write 

\\S'k((3)  +  U(/3)\\   <   \\Uk((3)-Uk((30)\\  +  \\Rk((3)-Rk({30)\\ 

+\\Uk((30)-U((30)\\  +  \\U((3)-U((30)\\ 

M\Rk(Po)\\ 
<  \\Uk((3)  -  Uk(p0)\\  +  \\R*ffl  -  Rk((30)\\ 

+f+§  + ],  (2.5.30) 

by  (B.l),  (2.5.22)  and  (2.5.29). 

In  order  to  show  that  the  first  two  terms  of  the  right  hand  side  of  (2.5.30)  can  be 
made  arbitrarily  small,  let 

(A,  _  m  v  f       ^K(/3))         ,        b"'(h(xj(3))  ) 

We  assume  that  ?%(/3)  is  continuous  at  (3  —  (30  and  ||A"XT||  is  finite.  Thus 
lEW^M-^W^xJW   <  ^K(/3)-^(/30)|x||XXr|| 

<  ^    (t«i,2)  (2.5.31) 

and 

I E  fel  x  ll{^(/3)  -  w3j((30)}XjxJ\\   <   \  £  |g|  x  |W3j(/3)  -  u,3i(/30)|  x  ||XXT|| 

<  (2-5.32) 
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with  probability  -4  1. 

Therefore,  for  sufficiently  small  8  and  large  k,  from  (2.5.30),  (2.5.31),  and  (2.5.32) 
we  get 

\\S'k((3)  +  U((3)\\   <   \\Uk((3)-Uk((3o)\\  +  \\Rk(0)-Rk((3o)\\ 

e     e  e 

1  fc 

i  * 

+rEll(^(/3)-^(/90))^Tll 
l  * 

+rE  l&l  x  11(^0)  -  w3j(Po))xjxJ\\ 
K  i=\ 

3e 
+  5 

„     2e     2e  3e 

<   2x  I  |  

15     15  5 


(2.5.33) 


Thus  (C.l)  is  proved. 


Proof  of  Theorem  2:  Given  e  >  0  Theorem  1  implies  that  there  is  a  k0  such  that,  for 
all  k  >  k0,  {/3k}  €  Bs  and  0k)  satisfies  (2.2.8)  with  probability  exceeding  (1  -  e). 
Now  by  the  Taylor  expansion, 

0  =  Sk0k) 

=   5*09o)  +  5i(/3J09fc-/3o)  (2.5.34) 

where  fa  €  Bs.  Therefore, 

P+lfci  Sk(30)  +  fci  Si 09 J -  /30)|  =  0]  =  1.  (2.5.35) 
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Again  as  in  Theorem  1, 

\\Sm  +  U((3)\\   <   \\-Uk((3)-Rk((3)  +  U((3)\\ 

<  \\uk{(3)  -  uifi)\\  +  m(0)  -         +  \\Rk(Po)\\- 

For  k  >  k0  and  sufficiently  small  5  by  (2.5.29),  (2.5.31)  and  (2.5.35)  the  terms  in  the 
right  hand  side  of  the  above  inequality  can  be  made  arbitrarily  small  with  probability 
->  1.  Therefore, 

Thus,  the  limiting  distribution  of  kll2(fik  —  f30)  is  the  same  as  that  of 
t/_1  (/30)/c1/2Srfc(/30)-  The  result  therefore  follows  from  the  assumption  in  the  state- 
ment of  the  theorem. 

Proof  of  Theorem  4:  At  first  we  note  that  the  conditions  (A.l)  and  (A. 2)  are  sufficient 
to  prove 

Sk(90)  ^  0.  (2.5.36) 

By  conditions  (B.l)  and  (A. 2) 


Wk(0o)  -»  W(00), 

as  k  -»  oo  with  W(0O)  being  positive  definite. 
Since 


Rk(0)  = 


{Rk((3)¥xP  0"xl 
0ixp  o 


therefore,  by  assumption  (2.5.29) 


Rk(0o)  ^  0. 


By  the  conditions  (B.l)  and  (B.2)  in  (2.4.17)  we  get 


(2.5.37) 


(2.5.38) 


Uk(0Q)  ->  U(0Q), 


(2.5.39) 
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where  U(0)  is  finite  in  Bs  and  is  continuous  at  60.  Furthermore,  (C.l)  and  (C.2) 
directly  imply 

sup\\UTk(0)-UTk(00)\\^0.  (2.5.40) 

6<EBS 

Now  from  (2.4.16)  by  (2.5.38)  and  (2.5.39)  we  get 

S'k(0o)  %  U(60)  (2.5.41) 

Now  we  prove  the  following  inequality  which  will  be  useful  in  proving  the  theorem 
later 

\\S'k(eo)Wk\0o){S'k(0)}T  -  U(0o)W-\8o)UT(O)\\ 

<  \\S'k{9o)W];1{0o)Ul(9)  -  Sl(9^?{4o)UZm\ 

+  \\S'k(0o)Wkl(0o)UTk(0o)  +  U(0o)W-l(0o)UTk(0o)\\ 
+  \\U(0o)W-l(0o)UTk(0o)  -  U(0o)W-1(0o)UT(0o)\\ 
+  \\U(0o)W-1(0o)UT(0o)  -  U(0o)W-l(0o)UT(0)\\ 
+  \\S'k(0o)Wk1(0o)Rk(0)  ~  S'k(0o)Wk1(0o)Rk(0o)\\ 
+  \\S'k(0o)Wk1(0o)Rk(0o)\\ 

<  ^kmmlm\  *  Mm  -  tffwii 

+  ||  -  Uk{0,))Wk\0,)UTk{0Q)  +  U(0o)W-l(0o)UTk(0Q)\\ 

+  \\Rk(0o)W;1(0o)UTk(0o)\\ 

+  \\U(0o)W-1(0o)\\  x  \\UTk(00)  -  UT(00)\\ 

+  \M0o)W-1(Oo)\\x\\UT(Oo)-UT(O)\\ 

+  \\S'k(0*)W?m\\  x  \\Rk(0)  -  Rk(80)\\ 

+  \\S,k(eo)Wk1(0o)Rk(0o)\\  (2.5.42) 

<  e,  (2.5.43) 
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with  probability  — >  1,  where  e  is  an  arbitrary  small  positive  number.  This  is  be- 
cause by  (2.5.36)  -  (2.5.41),  all  the  terms  in  (2.4.41)  can  be  made  arbitrarily  small 
for  large  k  and  sufficiently  small  S  with  probability  — >  1.    Now  we  define  r  = 

i/(4\\{u(eo)w-1(e0)uT(e0)}-1\\), 

Tk  =  l/(4\\{S'k(0o)W;1(0o){S'k{Oo)}T}-l\\)  and  select  5  sufficiently  small  <E  Bs  so 
that 

\\u(e0)w-1(e0)uT(e)  -  u(e0)w-1(e0)uT(e0)\\  <  r/2  (2.5.44) 

and  (2.5.43)  hold.  Using  (2.5.43)  we  get  rk  -» ■  r  in  probability.  Thus,  by  (2.5.43)  and 
(2.5.44), 

\\s,k(o0)Wj;1(o0){s'k(e)}T  -  s'k(eo)w-k\eo){s'k(0o)}T\\ 

<  \\S'k(e0)W-k\00){Sk(0)}T  -  U(00)W-\Oo)UT(d)\\ 
+   \\S,k(Oo)Wk-1(0o){S'k(eo)}T  -  U(do)W-1(do)UT(0o)\\ 

+  \\u(eQ)w-1(e0)uT(e)  -  u(eo)w-1(0o)uT(0o)\\ 

<  t  <2rk  (2.5.45) 

with  probability  — >  1  as  k  — >  oo.  Now,  using  Inverse  Function  Theorem  (Rudin, 
1976),  with  probability  —>  1, 

S*k(0)  =  Sk(0Q)Wkl(0o){Sk(d)}T  (2.5.46) 

is  one-to-one  function  from  Bs  to  S*k(Bs),  and  S*k(Bs)  contains  the  open  circle  of 
radius  rk6/2  about  S*k(0o).  Also,  by  (A.l)  and  (A.2),  0  €  Sk(B6)  with  probability 
-4  1  as  k  qo.  Which  implies  that  0  e  S*k(B5)  with  probability  ->  1  as  k  oo. 
Consider  the  inverse  function  S*^1  :  S*k(Bg)  Bs.  It  is  well  defined  whenever  S£ 
is  one-to-one,  i.e.,  with  probability  ->  1.  Since  0  6  S*k(Bs)  with  probability  -4  I'  M 
k  — V  oo,  we  conclude  that: 

1.  the  root,  ^_1(0),  of  (2.2.11)  exists  in  Bs  with  probability  ->  1  as  k  -*  oo; 
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2.  since  6  may  be  taken  arbitrarily  small,  S*k  l(0)  converges  in  probability  to  6q; 

3.  Since  S*k  and  B$  are  one-to-one,  any  other  sequence  {9~k}  of  roots  of  (2.2.11) 

necessarily  lies  outside  Bs  with  probability  — >  1,  i.e.  the  sequence  does  not 
converge  to  90. 

The  proof  is  now  complete  with  Ok  —  5^_1(0). 


CHAPTER  3 


CONDITIONAL  AND  UNCONDITIONAL  BOOTSTRAP  STANDARD 
ERRORS  AND  CONFIDENCE  INTERVALS  OF  THE 
EMPIRICAL  BAYES  ESTIMATORS 


3.1  Introduction 

In  the  last  chapter  we  introduced  an  empirical  Bayes  approach  in  model  based 
finite  population  sampling  theory.  We  showed  that  empirical  Bayes  estimates  perform 
well  in  a  decision  theoretic  sense.  However,  as  discussed  earlier,  standard  errors 
calculated  from  the  estimated  posterior  variance,  are  typically  too  small,  leading  to 
confidence  intervals  that  are  too  short.  In  this  chapter  we  introduce  a  comprehensive 
methodology  to  deal  with  the  problem  of  correctly  computing  confidence  intervals 
and  standard  errors  of  EB  estimates.  Here  also  we  assume  the  superpopulation  as 
the  natural  exponential  family  (NEF)  with  quadratic  variance  functions  (QVF)  and 
conjugate  priors  (CP)  for  the  mean  parameters.  We  consider  model  given  in  (2.2.2) 
with  (j>j  =  1  for  all  j  =  1,  . . . ,  k.  In  the  small  area  setting  k  is  assumed  to  be  large. 
As  before,  we  model  the  prior  means  with  general  link  functions,  i.e.  g(rrij)  =  xj(3, 
where  Xj  is  the  vector  of  explanatory  variables  for  the  j  th  stratum,  and  (3,  and  A 
are  the  hyper-parameters. 

The  inferential  version  of  the  model  is 


Uj    ~d  Marg 


V(m.i)  A  -I-  n, 
TUj,  — 5 — —  x  - 


rij        A  — 1»2 
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where  ^  =  (1  —  jBj)y7  +  B^rn,  is  the  posterior  mean  and  Bj  =  A/(A  +  rij). 

The  EB  estimate  is  given  by  fij  =  (1  —  Bj)y~j  +  Bjfhj,  which  we  get  by  just  plugging 
in  the  estimates  of  (3  and  A  in  the  posterior  mean  «?,  The  naive  standard  errors  are 

calculated  as  J  ^+n'_v  y  and  are  typically  too  small. 

In  the  above  EB  model  we  propose  a  more  direct  approach  to  the  standard  error 
calculation  problem.  With  (ij  as  the  EB  estimate  of  the  j  th  mean,  we  try  to  estimate 
the  distributions 

\Hj-fij\yj)  and  [nj  -  fa], 

using  a  parametric  bootstrap  procedure.  The  reason  we  call  it  a  direct  method  is, 
unlike  Carlin  and  Gelfand  (1991)  we  do  not  calibrate  a  naive  EB  interval.  Rather  we 
estimate  the  conditional  distribution  as  done  by  Hill  (1990)  in  the  normal-normal  EB 
model.  We  condition  on  yj  as  done  by  Carlin  and  Gelfand.  Thus  our  methodology 
is  a  combination  of  the  ideas  of  Carlin  and  Gelfand  (1991)  and  Hill  (1990).  As  we 
have  already  noted  that  the  posterior  distribution  of  \i$  \  y  depends  on  the  data  only 
through  y~j.  Therefore,  conditioning  on  y~j  seems  to  be  the  right  thing  to  do  in  the 
absence  of  an  ancillary  statistic. 

The  bootstrap  estimate  of  the  unconditional  and  conditional  distributions  are 

[//*  -  ft*  |  A,  0}  and  [fj,*  -  fi*  \  y~j,  \,(3]. 

Thus,  the  bootstrap  estimate  of  the  mean  squares  errors  are 

E[{^-^}2\\J]  and  E[{^-^}2|^,A,^], 

In  Section  3.2  of  this  chapter,  we  show  the  way  to  calculate  the  Monte  Carlo  esti- 
mates of  the  conditional  and  the  unconditional  bootstrap  standard  errors  of  the  EB 


Mi 


(A  +  rij  -  v2) 


(3.1.1) 
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estimates.  The  methodology  was  applied  to  estimate  the  bootstrap  standard  errors 
of  the  EB  estimates  found  for  the  23  different  lifetime  occupational  categories  in  the 
cognitive  impairment  problem. 

In  Section  3.3  compare  the  conditional  and  unconditional  asymptotic  distributions 
of  the  standardized  EB  estimators.  Carlin  and  Gelfand  (1986)  computed  conditional 
confidence  intervals  associated  with  EB  estimators  but  did  not  provide  any  mathemat- 
ical justification  for  their  conditioning.  From  Theorem  1  one  can  see  that  conditional 
and  unconditional  distributions  of  /ij  —  fij,  for  j  =  1,  . . .,  k,  are  asymptotically  the 
same  for  the  normal  sampling  distribution  with  a  normal  prior.  In  Theorem  2,  for  the 
EB  problem  with  NEF-QVF  sampling  distribution  with  a  conjugate  prior,  we  show 
that,  if  the  overdispersion  parameter  A  is  known,  then  unlike  the  normal-normal  case 
the  conditional  distribution  \fij  —  fij  \  jjj]  and  the  unconditional  distribution  [/i,  —  fij] 
have  two  different  limiting  distributions.  Here  along  with  the  assumption  of  k  — >  oo, 
we  further  assume  that  the  rij  are  of  order  k.  In  Theorem  3  we  prove  the  same  result 
under  the  assumption  that  both  (3  and  A  are  unknown.  The  proofs  of  all  the  theorems 
are  given  in  Section  3.4. 

3.2    Computation  of  Conditional  Standard  Errors  and  Confidence  Intervals 

In  this  section  we  provide  the  methodology  to  compute  unconditional  and  condi- 
tional standard  errors  of  the  EB  estimators  from  the  NEF-QVF  model  with  a  con- 
jugate prior  distribution.  With  fij  as  the  EB  estimate  of  the  j  th  mean,  we  try  to 
estimate  the  distributions 

bh  -  N 1 9i\  and     ~  ihl 

by  using  a  parametric  bootstrap  procedure.  The  bootstrap  estimate  of  the  uncondi- 
tional and  conditional  distributions  are 

[fi*-fi*\\,0]  and 
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Thereby,  the  bootstrap  estimate  of  the  mean  squares  errors  are 

£[{/i*-/i*}2|A,0]  and  E[{$  -  A*  }2  |  fc, 

We  also  want  to  construct  the  bootstrap  estimate  of  a  two  sided  conditional  confidence 
interval  of  //,  .  Let  us  denote  that  and  q\-v/2  are  the  3  and  1  —  ?  percentile  points 
from  the  above  bootstrap  conditional  distribution.  Then  the  (1  —  if)  level  two  sided 
bootstrap  conditional  confidence  interval  is  given  by 

(Aj  +  9i»/2i  Aj  +  91-77/2)  • 

Now  since  the  above  conditional  and  the  unconditional  bootstrap  distributions  are  in- 
tractable analytically,  we  approximate  these  bootstrap  distributions  by  Monte  Carlo 
simulation.  We  generate  data  from  the  j  th  conditional  distribution  using  the  follow- 
ing scheme 

la   w  simulate  r        i     /   -i     simulate     r  _     ?     /  o 

calculate  ^  fc  ^  ^  ^  fc  ^  ^ 

ca'^ate  Ai  =  (1  -       +  4% 

caic4ate    ^  -/V  (3.2.2) 

The  Monte  Carlo  approximation  of  the  distribution  [fij  —  fij  \  yj,  \,(3]  is  obtained  by 
following  the  scheme  in  (3.2.2)  a  large  number  of  times  (say  C).  Here  C  denotes  an 
integer  of  size  at  least  a  thousand.  Since  /9  and  A  are  unknowns  to  begin  with  we 
use  instead  the  bootstrap  estimate  of  the  above  distribution;  that  is  we  plug  in  the 
estimates  (3  and  A  from  the  observed  data  in  the  above  scheme.  Thus 

P      -  A  J  <  x  I  yjt  {3,  A )  m  i  £  ifa  -  flfc  <x),  (3.2.3) 

0  i=i 

where  right  hand  side  is  the  Monte  Carlo  approximation.  From  the  Monte  Carlo 
approximation  of  the  bootstrap  version  of  the  conditional  distribution  given  in  (3.2.3), 
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we  calculate 

1  c 

to  find  out  the  Monte  Carlo  estimate  of  the  bootstrap  conditional  mean  square  error. 

In  order  to  find  Monte  Carlo  estimate  of  the  3  th  quantile  from  the  estimated 
distribution  we  order  the  (pi*  —  p\*  )'s,  obtaining 

Let  int(x)  denotes  the  integer  part  of  the  real  number  x  and  define 

intc{-)  =  min  [max{l,  int(-)},  C}. 

Then  (pi*  —  1S  our  Monte  Carlo  approximation  to  qv/2,  where  k\  =  intc{{C  + 

1)|}.  Indeed,  (pi*  —  — ►  9r//2  with  probability  one,  conditional  on  yj,  X  and  ft, 
as  C  goes  to  oo.  In  practice  it  is  common  to  chose  C  such  that  (C+1)S  is  exactly  an 
integer,  k\.  So  the  Monte  Carlo  approximation  of  the  lower  confidence  interval  for  pij 
is  frj  +  (pi*  —  pij Similarly,  for  the  upper  confidence  interval,  (pi*  —  pij)(k2)  is  our 
Monte  Carlo  approximation  to  qi-v/2,  where  k2  =  intc{(C  +  —  §)}.  Hence,  the 
Monte  Carlo  approximation  to  the  (1  —  77)  conditional  bootstrap  confidence  interval 
is 

{Pi  +  K  ~  £J)(*i)»  Aj  +  (mJ  -  AJ)t*a)  )■ 

We  calculate  the  conditional  and  unconditional  standard  errors  of  the  EB  estimates 
for  the  23  occupational  categories  from  the  "cognitive  impairment"  problem  (Table 
2.1).  In  this  problem  the  descriptive  version  of  our  EB  model  is 

_   ind  1  •    /  \ 

rijyj  ~  bin(nj,pij) 
with  the  conjugate  prior  is  a  beta  distribution.  Thus, 

pij  ~  beta(m\,  (1  -  m)A) 
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for  j  =  1,  . . .  ,  k.  We  model  marginal  mean  m  as 

logit(m)  =  p. 

The  estimation  procedure  for  estimating  (ra,  A)  is  given  in  Section  2.2.  We  have  also 
noted  that  the  naive  standard  errors  of  the  EB  estimates  are  typically  too  small  as 
they  fail  to  account  for  the  estimation  of  parameters.  Thus  we  prefer  to  use  cal- 


culate the  conditional  and  unconditional  standard  errors,  \jE[{iJ,j  —  fij}2\yj,  m,  A] 
and  ^E[{fj,j  —  /ij}2|m,  A],  where  fij  is  the  EB  estimate  of  j  th  mean  fij.  Boot- 


strap estimates  of  yjE[{nj  -  fij}2\m,  A]  and  ^E[{nj  —  fij}2\yj,  ra,  A]  are  given  by 

^[{^-/i*}2|m,A]  and  yj E[{^  -  fi*}2\  yj,rh,X]  respectively.  Since  these  expec- 
tations not  available  in  closed  form,  we  use  a  Monte  Carlo  simulation  techniques  to 
approximate  them.  The  results  are  given  in  Table  3.1.  The  last  two  columns  present 
the  Monte  Carlo  approximation  to  the  bootstrap  estimates  of  the  unconditional  and 
the  conditional  standard  errors  of  the  EB  estimates.  We  note  that  the  conditional 
and  the  unconditional  standard  errors  are  very  different  for  all  of  the  occupational 
categories.  This  result  is  in  agreement  with  the  theoretical  results  to  be  proved  in 
Section  3.3. 

As  a  special  case  of  Theorem  3  in  Section  3.3,  with  rij  —  O(k),  the  conditional 
distribution 

{  \A^(Aj  -  Mj)  I  V)  }  ~  N(  0,        -  yj) ) 
as  k  — >  oo.  On  the  other  hand  the  unconditional  distribution 

as  k  ->  oo,  where  F  is  the  c.d.f.  of  a  non-normal  random  variable.  This  difference  in 
the  asymptotic  distributions  is  the  key  factor  for  getting  very  different  values  in  the 
last  two  columns  of  Table  3.1. 
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Furthermore,  in  most  of  the  categories  the  conditional  standard  errors  are  slightly 
larger  than  the  naive  standard  errors  presented  in  Table  2.1. 

Table  3.1: 

EB  estimates  and  the  estimated  s.e.  of  the  means  from  the  cognitive  impairment  data 


Lifetime 

Sample 

Sample 

EB 

Uncond. 

Cond. 

occupation 

size 

prop. 

est. 

s.e. 

s.e. 

1 

Farm  managers 

383 

.380 

.377 

.0181 

.0247 

2 

Housewives 

369 

.317 

.315 

.0190 

.0242 

3 

Domestic  service  employees 

323 

.365 

.361 

.0201 

.0263 

4 

Skilled  workers 

319 

.255 

.254 

.0210 

.0241 

5 

Shopkeepers 

287 

.171 

.172 

.0216 

.0220 

6 

Unskilled  workers 

275 

.354 

.350 

.0226 

.0282 

7 

Public  white  collar  workers 

232 

.162 

.163 

.0237 

.0241 

8 

Craftsmen 

228 

.257 

.255 

.0250 

.0288 

9 

Farm  workers 

210 

.541 

.529 

.0267 

.0335 

10 

Private  middle  executives 

187 

.075 

.079 

.0254 

.0199 

11 

Private  white  collar  workers 

181 

.091 

.095 

.0272 

.0213 

12 

Business  employees 

136 

.135 

.138 

.0324 

.0284 

13 

Private  executives 

135 

.061 

.068 

.0316 

.0215 

14 

Policemen,  soldiers 

98 

.062 

.071 

.0362 

.0259 

15 

Primary  school  teachers 

68 

.060 

.073 

.0434 

.0298 

16 

Public  executives 

61 

.068 

.082 

.0461 

.0399 

17 

Drivers 

60 

.241 

.236 

.0461 

.0539 

18 

Public  middle  executives 

55 

.017 

.038 

.0439 

.0252 

19 

Teachers,  lecturers 

48 

.000 

.026 

.0487 

.0233 

20 

Nurses 

43 

.171 

.175 

.0497 

.0537 

21 

Professionals 

30 

.138 

.149 

.0626 

.0596 

22 

Other  occupations 

25 

.080 

.106 

.0634 

.0533 

23 

Inactives 

20 

.450 

.382 

.0702 

.0935 

3.3    Importance  of  Conditioning:  Mathematical  Justification 

In  this  section  we  prove  results  comparing  the  conditional  and  unconditional 
asymptotic  distributions  of  the  standardized  EB  estimates.  In  Theorem  1,  of  this 
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section  we  consider  the  simplest  EB  model,  with  normal  sampling  distribution  with 
normal  prior,  where  the  sampling  variances  are  known  and  equal.  In  this  model  an 
exact  EB  ancillary  statistic  exists.  As  we  have  discussed  before,  there  are  gave  several 
reasons  for  conditioning  on  the  ancillary  statistics  when  they  are  available.  However, 
Theorem  1,  shows  that,  at  least  to  first  order,  unconditional  and  conditional  EB 
inferences  are  the  same. 

Consider  the  following  empirical  Bayes  model 

Vjlfij  ~d  N(fij,a2), 

with  prior  distribution 

fij  ~d  N{fi,  r2), 

for  j  =  1,  . . .,  k.  Here  a2  is  assumed  to  be  known.  The  EB  estimate  of  the  j  th 
population  mean  is  fij  =  (1  —  B)y~j  +  By,  where  B  =  ^fea  ■  Then  the  following  result 
holds. 

Theorem  1: 


\y/a*(l-B)J 


hmP  [//.,-  Aj  <  t|«<  Vj  <u  +  8]   =  $  [  -T=J===  ]  +  0(k-'+e), 


and 


limP[^  -Aj  <*|u<  aj  <u  +  6]   =   f-f-7—ii      =)  +  0(k-i+% 
f~*°  \y/a*(l-B)J 

where  e  €  (0,  4)  can  be  arbitrarily  close  to  1/2.  Here  a,  =  jgft~f —  is  the  EB 
ancillary  statistic  and  $(•)  is  the  c.d.f.  of  standard  normal  distribution. 


50 


The  proof  of  the  theorem  is  deferred  to  Section  3.4.  Prom  the  Theorem  1,  it  fol- 
lows that  the  unconditional  distribution  and  the  conditional  distributions  of  the  EB 
estimates  in  the  normal-normal  EB  model  are  same,  at  least  to  first  order.  Further- 
more, they  all  have  approximately  the  same  normal  distribution.  Hill  (1990)  argued 
in  favor  of  conditioning  on  ancillary  statistic  aj  while  computing  confidence  intervals 
of  the  EB  estimate  fij.  From  Theorem  1  we  see  that  conditional  and  unconditional 
confidence  intervals  and  standard  errors  are  going  to  be  asymptotically  equivalent. 

In  the  next  two  theorems  we  prove  that  the  opposite  holds  for  nonnormal  natural 
exponential  family  sampling  distributions  with  conjugate  priors.  Here,  there  does  not 
exist  an  exact  ancillary  statistic.  Thus,  we  compare  the  unconditional  distribution  of 
the  EB  estimator  of  a  stratum  mean  with  its  conditional  distribution,  conditioning 
on  the  sample  observation  corresponding  to  that  stratum.  In  Theorem  2  we  consider 
the  dispersion  parameter  in  the  prior  distribution  to  be  known,  while  in  Theorem  3 
we  consider  all  the  hyper-parameters  to  be  unknown. 

Consider  the  EB  model  as  given  in  (3.1.1).  Under  the  assumption  of  known  A  the 
EB  estimate  of  the  j  th  population  mean  is     —  (1  -  Bj)yj  +  Bjfi,  where  Bj  = 

/}  =  ZJjLi  Wjfij  and  Wj  =    k  J — -.  Then  we  have  the  following  theorem. 
Theorem  2:  Assume  Cx  <  ^  <  C2,  and  k  ->•  oo.  Then 

YlmP[^(^j-fij)<t\u<yj<u  +  6}  =  ^(^L=)  +  0{k~ll2)  (3.3.4) 

\^V{u)J 

and 

Pl^TMi-h)  <*]  =  F*j  f  y    *   i  )  +  O(k-^whn),  (3.3.5) 
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where  r  >  0,  $(•)  is  the  standard  normal  c.d.f.,  and  Fk  ,•(•)  is  the  c.d.f.  of  v^"(%  ^  . 
In  special  cases  it  can  be  shown  that 

as  A;  — >  oo,  where  F  is  the  c.d.f.  of  a  certain  non-normal  random  variable. 

The  proof  of  the  Theorem  2  is  given  in  Section  3.4  of  this  chapter.  Comparing 
(3.3.4)  and  (3.3.5)  one  can  see  that  there  is  an  asymptotic  difference  between  the 
unconditional  and  conditional  distributions  of  fij  —  fij. 

We  now  consider  the  case  where  A  is  unknown.  The  EB  estimator  of  the  j  th 

population  mean  is  fij  =  (1  —  Bj)y~j  +  Bjft,  where  Bj  = 
Theorem  3:  Under  the  assumptions  of  Theorem  2, 

lim  P  [  ^(/ij  -fij)  <  t\u  <  y3 <  u  +  5]  =  $  (^-j==}j  +  o(l),  (3.3.6) 

and 

P[v**0*i  -  Af)  <*]  =  Fkj  (  .\  A  )  +  o(l).  (3.3.7) 

Fkj,  $,  are  same  as  defined  in  Theorem  2.  Here  also  in  special  cases  it  can  be  shown 
that  Fjtj  — >  F  ,  as  k  — >  oo,  where  F  is  the  c.d.f.  of  certain  non-normal  random 
variable.  We  now,  consider  two  important  special  cases,  which  are  widely  used  in 
practice. 

Case  1: 

Suppose  yjifij  ~  Poi(nj)  x  A  and  ftj  ~  g(lij),  where  g(fi})  =  $$e~<*iP$~l. 
In  this  case  E(y~j)  =  Var(yj)  =  fi. 
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Let  Xn.  =  V^fo  N\  Then  the  characteristic  function  of  Xn.  is  given  by 


<j>n.(t)  =  E(eitx»i  )  =  EE 


it 


Now 


where  Wn.(t)  =  nAe^*  1         |.  Hence, 


Using  the  fact  that  Ci  <  n^/A;  <  C2  we  get  lim^oo  Wnj(t)  —  —  fc.  Thus  the  charac- 
teristic function  of  the  limiting  distribution  F  is 


lim  </>„  (t)  = 

S— XX)  J 


1  + 


2/K1 


If  p  is  a  positive  integer  this  is  the  characteristic  function  of  the  convolution  of  p 
double  exponential  random  variables  with  common  p.d.f. 


/(Z)  =  ^ae>l2l. 


If  p  is  not  an  integer,  the  same  can  not  be  said. 


53 


Case  2: 

Suppose  yj\6  ~  Bernoulli  (6)  for  j  =  1,  . . .,  k,  and  9  ~  Beta(Xfi,  A(l  —  //)). 
Writing  X;t  =     ^{y-0)    |  one  can  show  that  the  characteristic  function 


'A+T 


4(*)  =  £(e<itx*>)  =  £ 


Now 


log 


-  -/kite  f  i  t 

8  V^1-*'^  x  |  (1  -  6)  +  9eV2k^-^jh 


VkitO 


y/Ml  ~  J) 
y/kite 


A 
A+l 


+  k  x  ZO0 j(l  -  0)  +  fcV^Wr  j 

^(i-m)^  l^Mi-M)^ 

Therefore,  using  the  bounded  convergence  theorem, 


lim  0fc(t)  = 

fc— >oo 


where 


0(0  =  J3 


exp<  -  -  r  x 


is  the  characteristic  function  of  the  limiting  distribution  function  F.  Thus,  in  this 
case  F  is  a  continuous  mixture  of  normal  random  variables. 

The  proof  of  Theorem  3  is  also  given  in  Section  3.4. 
Corollary  1:  In  Theorem  3,  if  the  sampling  distribution  is  normal,  that  is  if  the  NEF- 
QVF  is  a  normal  distribution  with  mean  /z,  and  known  but  unequal  variance  <r2/nj> 
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then  the  conjugate  prior  is  also  normal.  In  that  case  the  unconditional  distribution 
of  /}  is 


P[v^J0% -ih)<t]  =  $    ,  +o(l), 


and  the  conditional  distribution  is 


limP[v^7(/ij-/ij)  <■*■]«<  ft ;<u  +  £]    =   $  (  -7=====  )  +  o(l). 


Corollary  1,  gives  us  the  same  result  that  we  have  from  Theorem  1.  Here  we  have 
relaxed  the  assumption  of  equal  sampling  variance  but  have  assumed  that  the  rij  are 
also  large  and  are  of  order  k.  Thus,  comparing  results  of  Theorem  1  and  Corollary 
1,  with  those  of  Theorem  2  and  Theorem  3,  we  conclude  that  there  is  distributional 
difference  between  the  approximate  distribution  of  the  EB  estimators  unless  the  EB 
model  is  a  normal-normal  one. 

3.4  Proofs 

Proof  of  Theorem  1:  Write  Ajk  =  §j(B  -B)  +  Bfi-  By.  Then 

hh-h^*\   -   P[fij-{(l-B)yj  +  Bfi}  +  {yj(B-B)  +  Bl^-By}<t} 
=   P[nj  —  {(1  —  B)y~j  +  Bfi}  +  Ajk  <  t] 
=   P[»3  "  {(1  -  B)yj  +  Bp)  +  AJk  <  t,  \Ajk\  <  bk } 
+  P\to  ~  {(1  -  B)yj  +  Bp)  +  Ajk  <  t,  \Ajk\  >  bk  ] 

<  P\N  ~  {(1  -  B)yj  +  Bp}  +  Ajk  <  t, \Ajk\  <  bk]  +  P[\Ajk\  >  bk ) 

<  Pfrj  ~  {(1  -  B)yj  +  B(i}<t  +  bk]  +  P[\Ajk\  >  bk } 


=  P 


Hi  ~  {(!  ~  B)yj  +  Bn]  <      t  +  bk 


ypil  ~  B)         '  y/o*(i  -  B). 


+  P[\Ajk\>bk] 
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=  »H  +   /  +P[\Ajk\>bk] 

-  *  (^(i.fl))  +  °(6fc)  +  P[,Ajfc|  >  bk]-  (3'4'8) 


Similarly 


>  P\nj  -  {(1  -  B)yj  +  By)  +  Ajk  <  t ,  \Ajk\  <  bk } 

>  P[N  -  {(1  -  B)yj  +  B»}<t-bk,  \Ajk\  <  bk  ] 

>  P\jij  -  {(1  -  B)vi  +  By}<t-bk}-  P[\Ajk\  >  bk) 


=  $ 


,     *      *)  +  0(h)  -  P[\^jk\  >  bk)  (3.4.9) 


Now  we  need  to  find  out  the  order  of  P[|Ajjt|  >  bk].  We  write 

\yj(B-B)  +  Bti-By\   =   \y,{B  -  B)  +  B{fi  -  y)  +  y{B  -  B)\ 

m   \(B  -  B)(fi  -  y)  +  M  ~  V)\ 
<    \(B  -  B)(y~j  -  y)\  +  \BQi-y)\ 

Hence, 

{\(B  -  B)%  -y)\<  bk/2}     H     {>  -y\<  bk/2) 

=»   {\yj(B-B)  +  Bn-By\<bk}. 

Thus, 

P[\Ajk\>bk] 

<  P[\(B  -  B)(y,  -  y)\  >  bk/2]  +  P[\y  -  y\  >  bk/2] 

<  22r  E{%  -  y)»  (B  -  B)*}/%  +  E\y  -  y\2r/b2kr 
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<  22r  E 


=  22r  E 


t.*  {kr3K-B 


Zifij  -  y)2 


bfr  +  o(k-)  x 


f_        _\  2r 

Vj  -  y\ 


2r 


=  0(1)  x  E 
<  0(1)  x  E 
=  0(1)  x  £ 


J 

\s/k 

[k 

-3) 

y/k- 

-  lXfc-i 

(k 

-3)2 

(k- 

(k 

-3)2 

x  Bf  x  6fe-2^  +  0(*"r)  x  £ 


-r\  ^  L-2r 


2r 


x  6fc-2r  +  0(ATr)  x  b~k 


-r\  w  L-2r 


2r 


x  fefc"2r  +  0(ATr)  x  b~k2r 


+  0(k-*)  x  fefc-2r 


J  V     (^-3)2  ; 


2ri 


x  ^2' 


4r \  1/2 


+  0(A;"r)  x  bf 
=   0(1)  x  Ml  -  2^]    |  7  x  6fc-2r  +  O(fc-)  X  bf' 

=  0(A;-r)x6,-2r 

=  0(AT2rf)   (with  bk  =  AT1/2+e). 


(3.4.10) 


In  the  third  inequality  in  (3.4.10)  we  defined  s2  =  J2(9j  ~  y)2/(k  -  1)).  The  second 
equality  in  the  right  side  of  (3.4.10)  is  because  is  independently  distributed  of 
s2,  and  E(^)2r  =  Eiy-j-y^/Eis2*)  =  0(1).  We  get  the  fourth  inequality  by  mul- 
tiplying the  positive  valued  random  variable  (         —  +  1)  within  the  expectation. 

Now  if  we  equate  -2re  =  e  -  \  then  e  =  I /{At  4-  2)  G  (0, 1/2),  and  can  be  made 
arbitrarily  small.  Thus  from  (3.4.10) 


P[\Ajk\>bk]  <  0(H+**r) 
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<  0(k~2+e) 


(3.4.11) 


Therefore,  from  (3.4.8),  (3.4.9)  and  (3.4.11)  we  get 


P[N-^<t}  =  <5>(-j= 


t 


2(1-5). 


+  0(k~>+t) 


(3.4.12) 


Before  approximating  the  conditional  distribution,  we  need  the  following  lemma. 

Lemma:!  In  the  above  normal-normal  problem  let  aj  =  (y~j  —  y)/s  be  the  empirical 
Bayes  ancillary  statistic.  If  aj  =  J*i£Ti ,  then 

P[\aj-a*\  >k-i+t]  <0{k~^) 


Proof: 


a>j  -  a*  |  < 


y- V 


+  \Vj  -  y\  x 


Hence,  using  Chebyshev's  inequality,  the  independence  of  a,  and  s2,  and  the  fact  that 
^2  ~  xLi,  one  gets 


P[\aj-  a*  |  >  2bk]   <  P 


y- v 


+  P 


yj  -  y 


>  bk 


< 


< 


M 


y  -  v 


2r 


+  E 


yj-y 


2r 


jo(ATr)  +  £( 
Ejyj  -  y)2r 


fc  -  y 


2r 


2r 


x  E[  1 


xb 


X  b~* 


-2r 


2r 


(2        \  'Jr 
i  -      j  +  o(it-r)  x  b 

=   0(k~2re);   (with  2bk  —  AT1/2+£ ). 
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Taking  e  =  l/(4r  +  2)  one  gets 

P[\dj -a*\  >  k~i+e}   <   0(fc-2  +  27TT) 

=  o(Ht«). 

Now  we  want  to  approximate  lim^o  P[^j  —  fa  <  t  \  u  <  jjj  <  u  +  6]. 

P[Hj  —  fa  <  t  \  u  <  yj  <  u  +  6] 

P[lij  —  fa  <t,  u  <  y~j  <  u  +  5] 
P[u  <y~j  <u  +  5] 

P[fij  -  {(1  -  B)yj  +  Bfi}  +  Ajk<t,u<yj<u  +  6] 
P[u  <  Vj  <  u  +  6} 

P[Hj  -  {(1  -  B)yj  +  Bfj,}  +  Ajk<t,u<yj<u  +  5,  \Ajk\  <  bk ] 

P[u  <y~j  <u  +  5] 

P\jij-{{1-B)yj  +  Bfi}  +  Ajk<t,  u<yj<u  +  S,  \Ajk\  >  bk] 

P[u  <yj<u  +  5] 

Now  using  the  same  argument  as  before  in  the  unconditional  case  we  get 
lim,j_>o  P  [fij  ~  fa  <t\u<y~j  <  u  +  6] 


<   Um  P[u  <y3<u  +  5,  %  -y\\B-B\>  bk/2) 
~   s^o  P[u  <y~j  <u  +  5] 

i  hmplu^y^u  +  ^  \y-»\>bk/2] 

<5->o  P[u  <y~j<u  +  5] 

I  ^^-{^-B^j  +  BfMyKt  +  bk,  u<yj<u  +  8] 

-5^0  P[u  <yj  <u  +  S] 


(3.4.13) 
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We  note  y~j  is  independently  distributed  of  \uj  -  {(1  -  B)y~j  +  B(jl}).  Therefore,  third 
term  of  the  right  hand  side  of  (3.4.13)  is  equal  to 

Pfa  -  {{I  -  B)$i+B»}  <t+hk\ 

Hence,  as  in  unconditional  case  this  is  equal  to 


+  0(bk). 


For  the  second  term  in  (3.4.13)  we  note  that  y  —  [i  =        +  —  A*)>  where 

V(-j)  =  ITi  SnyW-  Since' 
w  <  ^  <  u  +  S,  \y  -  u\  >  bk/2 


u 


^-^  .  k  —  1 ,  .  ,  tn  u  +  S  —  u 
u<Vj<u  +  8,  —^-{y^  -  u)>  bk/2  -  


^-^  ?  k—l,_  .  u  +  S  —  u 
u<Vj<u  +  6,  ——(y^j)  -  u)  <  -bk/2  


the  second  term  in  (3.4.13) 


faPl«<frp+Mfr-M>W2|  <  lim(P 

s^o  P[u  <y~j<u  +  5]  ~  6->o  [ 


k-1 
k 


(V(-j)  -l*)>  h/2  - 


u  +  S 


+  P 


k-1  u+S—u 
—  (y(-j)  -/»)■<  -bk/2  ^— £ 


Writing  u*  =  ^"r+ra  and  s*  =  jJ+T*  the  denominator  of  the  first  term  in  (3.4.13) 
can  be  written  as 

P[u<yj  <u  +  S]    =   P[u*  <  a*  <  u*  +  8*] 

>   P[u*<  a*  <u*  +  8*,  |a*  -  a,  |  <  k~l/2+e } 
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>  P[u*  +  k~1/2+e  <  aj  <u*  +  6*-  AT1/2+€,  \a*  -  aft  <  k~1'2^ } 

>  P[u*  +  AT1/2+£  <  aj  <u*  +  8*-  k~1/2+e] 
-P[\a*-aj\>k-^}. 

The  numerator  of  the  first  term  in  (3.4.13)  can  be  written  as, 
P[u<yj  <u  +  8,  \yj-y\  \B-B\>bk/2] 

=   P[u*<a)<u*  +  8\\yj-y\\B-B\>bk/2} 

=  P  [«*  <  oj  <  u*  +  8*,  \§i  -y\\B-B\>  bk/2,  \a*  -  aj\  >  k~1/2+t } 

+P  [u*  <  a*  <u*  +  8\  \yj-y\\B-B\>  bk/2,  \a*  -  dj\  <  k~ll2+t  ] 
<   P  [u*  -  AT1/2+£  <  aj  <u*  +  8*  +  AT1/2+£,  \yj  -y\\B-B\>  bk/2 } 

+P  [\a*  -  a,j\  >  A;"1/2+e  ]. 

Hence,  first  term  in  (3.4.13) 

=   Hm  p[u  <yj<u  +  8,  \y,  -y\\B-B\>  bk/2) 

s^o  P[u<y-j<u  +  8]  \  ■  ■  ) 

<  P  [u*  -  k~V2+c  <  a,  <u*  +  8*  +  A:~1/2+t,  jjfr  -y\\B-B\>  fe] 

-   SK  P  [u*  +  k~ll2+e  <  a,  <u*  +  8*-  jfe-1^*]  _  p[\a*  -aj\  >  k~l/2+( } 


+  lim 


P[\a*-aj\  >  k~1/2+e] 


«-*>  P  [u*  +  k~ll2+i  <  aj  <u*  +  8*-  fc-V2+«]  _  p[\a*  -aj\>  k~1^  ] 

P  [u*  -  k~^  <aj<u*  +  8*  +  k'^,  \yj  -  y\  \B  -  B\  >  % }  n(,.1/2+e. 
s^o  p  [u*  +  <  ttj  <u*  +  8*-  fc"1/2+€]  +  W  > 


<   lim  |p[(u*  +  8*  +  k-V2+t)  x\B-B\s>  bk/2] 


x  P^-k-^^a^u'  +  P  +  k-V^] ) 
P  [u*  +  fc-i/a+«  <  aj  <u*  +  8*-  Jfc-i/a+e]  J     *^\™  ; 

<   lim  (p  [(u*  +  8*  +  k~1/2+()  x\B-B\s>  bk/2 } 
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P  [u*  -  2k-^  <g*<u*  +  6*  +  2k-V*«]  ]  _1/84< 
P  [u*  +  2k~l l2^  <  a)  <u*  +  8*-  2ArV2+<]  j  ^    V  J 

<   lim  |p      +  5*  +  AT1/2+f )  x  \B  -  B\ s  >  bk/2 } 

$(u*  +  2k~^  +  6*)  -  $(u*  -  2k~^)  \  n(k-1/2+e) 
$(w*  -  2Ar1/2+£  +  5*)  -        +  2A;-1/2+f)  J  +    ^  ' 

=  0(6*)  x  lim  (1  +  0(6*))  +  0(k~1/2+t) 

=  o(r1/a+t). 


(3.4.15) 


The  second  equality  in  (3.4.15)  follows  by  Lemma  1.  The  second  inequality  is  due  to 
the  independence  of  sufficient  statistic  and  ancillary  statistic.  The  third  equality  in 
(3.4.15)  follows  from  the  same  argument  as  used  in  Lemma  1.  Thus,  the  conditional 
distribution 

lim  Pfa  -  fij  <t\u<yj  <u  +  6]<<f>  ^     =J    g*j  +  0(AT1/2+e),  (3.4.16) 

where  e  is  an  arbitrarily  small  positive  number.  Now  using  the  same  argument  as 
used  to  prove  (3.4.9)  we  get 

lim  P[fij  -  %  <  1 1  u  <  yj  <  u  +  S]  >  $  ^     =J    ^  +  0(k-^2+e).  (3.4.17) 

Thus  from  (3.4.16)  and  (3.4.17)  we  get  the  desired  result,  namely 

lim  Pfo,  *  A,-  <  1 1  «  <f4  <  u  +  £]  a  *  f   y     *         ]  +  0(A;"1/2+e).  (3.4.18) 

\y/a2(l-B)J 

Now  we  want  to  approximate  limtf_>0  -  #j  <  <  I  «  <  Oj  <  «  +  5],  where  is  the 
EB  ancillary  statistic  as  defined  before. 
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P[lXj  —  fij  <  t  \  u  <  dj  <  u  +  5] 

P[fXj  —  p,j  <t,  u  <  cij  <  u  +  8] 
P[u  <aj<u  +  6] 

P[Hj  -  {(1  -  B)yj  +  By}  +  Ajk<t,  u<aj  <u  +  5] 

P[u  <  Qj  <  U  +  S\ 

P[Hj  -  {(1  -  B)yj  +  Bn}  +  Ajk<t,u<aj<u  +  8,  |A#j  <  bk } 

P[u  <aj  <u  +  S] 

|  P\fij  -  {(1  -  B)yj  +  Bn)  +  Ajfc  <  t ,  u  <  aj  <  u  +  6 ,  [Ajfc|  >  bk  ] 

P[u  <aj  <u  +  8] 

Using  the  same  argument  as  used  in  the  unconditional  case,  we  get 


P[fij  —  fij  <  t  \  u  <  aj  <  u  +  5] 

U    P[u  <aj<u  +  5,  %  -y\\B-B\>  bk/2] 
P[u  <aj<u  +  S] 

|  P[u  <aj<u  +  6,  \y  -  n\  >  bk/2] 
P[u  <cij<u  +  6] 

|  P[fij  -  {(1  -  B)yj  +  Bn)  <t  +  bk,  u  <  aj  <  u  +  S] 
P[u  <  a,j  <  u  +  6] 

<    P[u  <aj<u  +  S,  %  -y\\B-B\>  bk/2] 
P[u  <dj<u  +  5] 

+P[\y-Li\>bk/2} 

|  P[Hj  -  {(1  -  B)y~j  +  By}<t  +  bk,  u  <  dj  <  u  +  5] 
P[u  <dj  <u  +  8] 

P[u  <  dj  <  u  +  6,  (u  +  6)  x  \B  —  B\  >  bk/2] 
P[u  <  dj  <  u  +  S) 

+0(k~r)  x  0(bk2r) 


P[pij  -  {(1  -  B)yj  +  B/j,}  <t  +  bk,  u<aj<u  +  5] 
P[u  <aj<u  +  8] 

=   P  [{u  +  8)  x  \B  -  B\  >  bk/2]  +  0{k~r)  x  0(bk2r) 

P[fij  -  {(1  -  B)yj  +  Bp}  <t  +  bk,  u<aj  <u  +  8] 
+  P[u  <aj<u  +  8] 

=  0(1)  x  0{b-k2r)  x  0{k~T) 

P[fij  -  {(1  -  B)yj  +  Bn}  <t  +  bk,  u<aj  <u  +  8] 
P[u  <  aj  <  u  +  8] 

=  0(1)  x  r2rf 

P\Hj  -  {(1  -  B)yj  +  Bn}  <t  +  bk,  u<aj  <u  +  8] 
P[u  <dj  <u  +  8] 

P[fij  -  {(1  -  B)jjj  +  Bfi}  <t  +  bk,  u<a,j<u  +  8] 
P[u  <aj<u  +  8] 

Now  we  need  to  find  out  the  order  of  the  last  term.  The  denominator  is 


P[u<aj<u  +  8] 

=   P  [u  <  ^  <  u  +  8,  \a*  -  aj\  <  k^2+e  ] 
+P  [u<aj<u  +  8,  \a*  -aj\>  k~1/2+e } 

>  P[u  +  k~1/2+t  <a*<u  +  8-  k~V2+e,  \a*  -  aj\  <  k~ll2+t } 

>  P[u  +  k-l'2+t  <a*<u  +  8-  k~1/2+e]  -  P[\a*  -0j\>  k~1/2+e } 
=   ft(«  +  8  -  k~1/2+€)  -  $(«  +  k-l'2+<)  +  0(k~l/2+e). 

The  numerator  is 
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P\pij  -  {(1  -  B)yj  +  Bn)  <t  +  bk,  u<aj  <u  +  6] 

-   P[fij  -  {(1  -  B)yj  +  Bix)  <t  +  bk,  u<aj<u  +  6,\a*j-aj\>  k~1/2+t ] 
+P\lb  -  {(1  -  B)yj  +  Bfj,}  <  t  +  bk  ,  u  <  a,  <  u  +  8,  \a*  -  aj\  <  AT1/2+£ } 

<  P  [fij  -  {(1  -  B)jfj  +  Bfj,}  <t  +  bk,  u-  k~1/2+t  <  aj  <  u  +  8  +  Ar1/2+£] 
+P[\a)-aj\>k-^] 

=   P      -  {(1  -  B)yj  +  Bn)  <t  +  bk]x  P[u-  k'1/2+t  <a*<u  +  8  +  k~1/2+e) 
+0{k-l'2+t) 

=   |$  ^     2J    g^j  +  0(6fc)|  x        +  <5  +  AT1/2+£)  -  $(u  -  fc_1/2+£)] 

+0(AT1/2+£).  (3.4.19) 


The  second  equality  in  (3.4.19)  is  due  to  the  independence  of  yj  and  \pj  —  {(1  —  B)yj  + 
Bfj],  and  by  Lemma  1.  Therefore,  the  last  term 


P[Hj  -  {(1  -  B)y3  +  Bn}<t  +  bk,  u<aj<u  +  S\ 
P[u  <  dj  <  u  +  S\ 


< 


Y/*2(l  -  B) 


) 


+  0(bk)  x 


$(u  +  8  +  k-l'2+t)  -  $(u  -  k~ll2+t) 
$(w  +  8  -  ArVz+e)  _  $(M  +  ifc-i/2+£) 


+0{k-1'2^) 


+0(Ar1/2+£) 


( 


*  B) 


+  0(6*)    x    1  +  0(A;~1/2+£)  x  {$(«  +  8)  -  $(u)}~ 
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+0(AT1/2+f) 

{K^)  +  0(Wh(l+0^+')x{1+^)) 

+0(Ar1/2+f) 


=  * 


^===j  +  0(6*)}  x  (l  +  0(k-1^)  x  {2  +  0(5)}) 
+0(AT1/2+£) 

|$  (^  3(1    J))  +  0(fc"1/2+£)}  x  x  {2  +  0(5)}' 

+0(fc-1/2+£),  (6*  =  fefV«+*). 


Therefore,  the  conditional  distribution 


limP^  -  fa  <  1 1  u  <  aj  <  u  +  5]  <  $  (   7==     ^= )  +  0(AT1/2+£), 


where  e  is  a  small  positive  number.  The  other  side  of  the  inequality  can  be  proved  in 
a  way  similar  as  (3.4.17).  Thus,  it  is  proved  that, 


lim        -  fij  <  t  \  u  <  aj  <  u  +  8]  =  $  [    ]  gJ         )  +  0(AT1/2+£). 


Proof  of  Theorem  2: 


lim^o  P  Iv^E)  (Mi  -  Aj)  <  *  I  w  <  j/j  <  m  +  5 ] 

=  J™  p  -  ft)  +  V^mBm  (% f -  A)  <  t \ u  <  y,  <  u  +  6} 

=  \™P[\/nm(Vj  -  Vj)  +  &jk<t\u<yj  <u  +  6] 
<   lim  P  [y/n^)(nj  -Vj)<t  +  nj(€k) \  u  <     <  u  +  5 } 
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+  JimP [|Ajfc|  >  n.(£fc)  \u<yj<u  +  6] 

+  lim P [|Aj-fc|  >  nT(efc)  \  u  <  yj  <  u  +  8],  (3.4.20) 

where  =  Jf^u^i &jik)  (j/j  —  A)-  From  Johnson  (Theorem  2.1,  1970)  we  get  the 
third  equality  in  (3.4.20).  He  studied  the  behavior  of  posterior  distribution  in  gen- 
eral, and  proved  that  properly  centered  and  scaled  posterior  distribution  function 
can  be  approximated  by  standard  normal  distribution  with  the  higher  order  terms  of 
order  n]$. 

Similarly,  it  can  be  shown  that 

lim^o P [y/nj{k)(l*j  ~  frj)  <t\u  <  yi  <  u  +  5] 

>  *  ( y==y)  +  °(nm)  +  °Kw)  ~  &  P  [lAjkl  >  n^ 1  u  -  9i  -  u  +  6 ]" 

(3.4.21) 

Now  we  find  out  the  order  of  the  last  terms  in  (3.4.20)  and  (3.4.21). 

lim^o  P[|Ajt|  >  nj{ek)  \  u  <     <  u  +  5} 

k 

E *>iVi\  >  0(k~e+12)  I  u  <  y3  <  u  +  6} 

3=1 

A*l  +  >  0(k~t+^)  \u  <  Qj  <  u  +  S] 


lim  P  [[ft  - 


<   lim  P  fly. 


67 


Km  P  ^  ~  ^  +  -  ^  >  Q(Art+^ ' M  -  % '  -  u  +  6  J 

«-+o  P  [  u  <  yj  <  u  +  8  ] 

=   lim  P(P«*J*r*|>  0(ATe+*)  -  |u  +  # -  ffl 

<  lim  P  [  £rt  £  Ik  -  Ml  >  0(fc_£+")  - f» 4 lr  Ml] 

<  lim    (C^k)2r  kE(Vi  ~  ^r 


< 


{0(Ar£+5)-  |u  +  5_/x|}2r 
Ar2r+1Q(1) 

/  \  2r 


=   0(AT3r+1+2re).  (3.4.22) 

For  the  first  equality  we  have  Bm  =  0(nj{k)),  nm  =  0(k),  and  t&j  =  ^n^/^+i^y 
For  the  second  inequality  we  have  Wj  <  C^/Cik.  Now  equating  —  3r  +  1  +  2re  =  —  e 
we  get  -e  =  -\-         Therefore,  from  (3.4.20),  (3.4.21)  and  (3.4.22) 

hmoF[^n^(/xj-/iJ)<i|u<yj<«  +  5]   =  $(y=y)  +  0(*T1/2) 


1      4r-3  , 


+  0(/i;~5~^+2) 


=  *(-?—]  +0{k-1'2). 


(3.4.23) 


For  the  unconditional  case,  proceeding  as  before  we  can  write 
PlV^miH  -Mj)  <  t] 

<  P  [y^W(Mi  -  fc)  <  * +  »g)  ]  +  P  [|Aifc|  >  nj{k) } 


68 


Similarly, 


(ff^-)  +  °(n^}  "  P  [|Ajfcl  >        ]-  (3-425) 


In  (3.4.24)  and  (3.4.25)  Pfcj  is  the  distribution  function  of 
Now 

P  [| Ajfc |  >  nj{ek) }   =  P  [y/nM)Bm \y3  -fx\  +  y/n^Bm\fi  -p\  >  n~{k)  ] 

<  p [V™mBm\yj  - /*l  >  n7(V 2 1  +  p \\/^mBm |£  -  /*l  >  ™7(V2 ] 
=  p  [  Ifj  -*|>  o(n-(J1/2)  ]  +  p  [  ia  -  n\  >  oinj^2) } 

<  E{yf) 
~  0(kr~2rt) 

=  0(k-r+2r£).  (3.4.26) 

Equating  -r  +  2re  =  -e  we  get  c  =  r/2r  +  1.  Therefore,  (3.4.24),  (3.4.25),  (3.4.26) 
implies 

-&)<*]  =  FkJ  )  ,\  i  )+0(k-1/2)  +  0(k^+^hn) 


7kJ  ,„*  i  )  +Olk-t+*£m.). 


V2 
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Proof  of  Theorem  3:  Proceeding  as  in  Theorem  2,  we  get 


Ply/nrnfaj  -in) 


Jv^)^)  +  0{nJil))  +  P  [|Ajfcl  >  n*k)  1  (3A27) 


and 


>  Fuji 


+  0(njil))-P[\A]k\>nj{k)],  (3.4.28) 


«2 


where  Ajk  =  y/nJ(h)Bj(u)(yj  —  A)-  In  this  case 
P[\^\>n-{(k)} 


<  P 


V^mBm\yj  -n\  > 


n 


m 


+  p 


n 


+  p 


[  (l+2fi)    >  2 


(3.4.29) 


Now  the  first  term  in  (3.4.29), 


agMlg£-fj  v.  "j(*) 


=  P 


>^,|A-A|>r 


+  P 


(TT5^)  >^-'lA-Al<^_ 


<   P[|A-A|>r]  +  P 


(1  +  ^)     >  2 


=   o(l)  +  P 


n 


lft-rt>-*gHi.+ 


-£-1/2  / 

iM  h  .  »j(*) 


A  +  r. 
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=  0(1)  +  — Ye!M — 

\      2       ▼  2(A+r)  J 

=  o(l).  (3.4.30) 

For  the  second  term  in  the  first  inequality  in  (3.4.30)  we  have  X  +  t>\>\  —  r. 
The  second  term  in  (3.4.29)  can  also  be  proved  of  small  order  by  a  similar  argument. 
Thus  it  is  proved  that 

P[y/*m{N  =  FkA    ,     *    i    1  +  o(l). 

For  the  conditional  case  the  first  few  steps  of  the  proof  are  same  as  in  Theorem  2. 
Mimicking  (3.4.20)  and  (3.4.21)  we  get 

lim^o  P  [y/nm  [fij  -  P-j)  <t\u<yj  <u  +  S] 

*  *(^))  +  °&>.  +  0(n^2) 

+  lim  P  [\Ajk\  >  nj{ek)  \u<yj  <u  +  6],  (3.4.31) 

and 

limj-K)  P  [y/nj{k){^i  ~  Pj)  <t\u  <y-j  <u  +  6] 


>  $1 


t 


-  P  [\Ajk\  >  n~{k)  |  u  <  yj  <  u  +  5].  (3.4.32) 
In  this  case  Ajk  =  ^/nJ^jBj{k)  {jfa  -  fi).  Hence,  using  d  <  rtj/k  <  C2, 
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lim^o  P  [|  Ajfc|  >  njk)  \  u  <  yj  <  u  +  8} 


<  limP 


n 


m 


u  <  yj  <  u  +  6 


+  limP 

<5->0 


u  <  yj  <  u  +  8 


=  limP 

<5->0 


(i  +  »)  2 


u  <  yj  <  u  +  8 


+  limP 

<5->0 


V^m  »J(fc) 
(TT^P)  2 


M  <  ?/j ;  <  U  +  8 


<  limP 

<5->0 


s/nrn\y~i  -  Ml  ^(fc) 


+  limP 

<S->0 


u  <yj  <u  +  8 


> 


tt  <  y j  <  u  +  8 


+  limP 

<s->o 


(l  +  =fi)  4 


w  <  ?/j  <  w  +  8 


The  first  term  in  the  right  hand  side  of  (3.4.33)  is 


(3.4.33) 


=  limP 

<5->0 


n 


> 


(l  +  »)  "  2|«  +  5-^| 


u  <  y j  <  u  +  8 


=  limP 

(S->0 


n 


y/nm 

[(1  +  ^fl)  '  2|«  +  <J-/i| 


> 


■;(*) 


,|A-A|>r 


u  <  yj  <  u  +  8 


+  limP 

(J->0 


A 

(1  +  2^*1)  -  2|u  +  *-/*l 


n 


> 


,|A-A|<r 


u  <  yj  <  u  +  8 


<   lim  P  [  |A  —  A|  >  t\u  <  y~j  <  u  +  8] 


+  limP 

<5->0 


> 


n7w 
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<   limP[|A  -  X\  >  t  |  u  <  yj  <  u  +  8]  +  0 

=  o(l).  (3.4.34) 

The  other  two  terms  in  (3.4.33)  can  also  be  proved  to  be  of  order  o(l)  in  a  similar 
fashion.  Therefore  using  (3.4.32),  (3.4.33)  and  (3.4.34)  we  get 

limP  [y/nJ^(Hj  -  Aj)  <t\u<yj  <u  +  S}  =  $^^1—^  +  o(l). 


CHAPTER  4 


MONTE  CARLO  APPROXIMATION  OF  BOOTSTRAP 
STANDARD  ERRORS 


4.1  Introduction 


In  this  chapter  we  use  asymptotic  methods  to  develop  rules  of  thumb  for  the  num- 
ber of  resamples  required  for  accurate  simultaneous  Monte  Carlo  (MC)  approximation 
of  bootstrap  standard  errors.  We  begin  by  briefly  reviewing  the  bootstrap  method 
for  standard  error  calculation  and  its  approximation  by  MC  techniques. 

Suppose  F  is  the  joint  distribution  function  for  the  random  vector  Y ,  and  we  are 
interested  in  estimating  the  mean  squared  error  matrix  of  the  estimate  (3,  of  the  k 
dimensional  parameter  /3, 


E  =  MSE{0 


0)  =  E{@-0)0-0f}. 


The  bootstrap  estimate  of  E  is 


t  =  E*{(0* 


0)0* -fif}  =  E*{<r<rT) 


where  E*  denotes  expectation  taken  with  respect  to  the  fitted  distribution  F  and  /3* 
is  the  version  of  /3  computed  using  a  resample  Y*  drawn  from  F.  The  Monte  Carlo 
approximation  to  the  bootstrap  covariance  matrix  E  has  the  form 
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where  d*  is  the  "deviation"  vector,  ((3*  —  /3)  based  on  the  j  th  resample  and  B  is  the 
number  of  resamples.  The  theoretical  bootstrap  covariance  matrix,  is  the  limit  of  the 
above  MC  estimate;  i.e.  Y,B  H>  £  as  B  — >  oo.  For  k  =  1,  we  denote  £  by  a2  and 
by  a2,,  where  by  definition  a  =  {E0  -  Z?)2}1/2. 

Efron  (1987)  addresses  the  question  of  the  number  of  resamples  required  to  accu- 
rately approximate  a  single  bootstrap  standard  error  (see  also  Efron  and  Tibshirani, 
1993,  section  6.4).  Efron's  analysis  targets  the  unconditional  coefficient  of  variation 
of  aB,  the  MC  approximation  of  a.  We  now  briefly  describe  Efron's  method. 

The  bootstrap  estimate  based  on  B  replications,  aB  =  {B~l  Z/f=1  dj2}1^2,  nas 
conditional  coefficient  of  variation  (standard  deviation  divided  by  expectation) 

CV*{aB)  app=x  [(8  +  2)/4B]1/2,  (4.1.1) 

where  8  is  the  kurtosis  of  the  fitted  distribution  F.  The  notation  indicates  that  the 
observed  Y  is  fixed  in  the  calculation  of  the  coefficient  of  variation  of  &b-  From  (4.1.1) 
we  can  see  that  as  B  — >  oo,  CV*{&b}  — >  0  and  thereby  og  a,  the  ideal  bootstrap 
estimate  of  the  true  standard  error  a.  However,  a  is  itself  subject  to  sampling  vari- 
ability. Let  CV{a}  be  the  unconditional  coefficient  of  variation  of  a.  Efron  (1987) 
proved  that  the  unconditional  coefficient  of  variation  of  aB  can  be  approximated  by 

CV{aB}  app=x  [CV2{a}  +  (MS  +  2)/AB}^2.  (4.1.2) 

So  according  to  his  criterion  in  (4.1.2)  the  variability  in  aB,  comes  from  two  different 
sources:  one  is  due  to  sampling  variability,  and  the  other  is  due  to  Monte  Carlo 
simulation.  Efron  used  this  criterion  to  determine  the  number  of  resamples,  B.  For 
example  under  the  assumption  of  ES  =  0,  and  CV{a}  =  0  (no  sampling  variability), 
we  find  CV{aB}  =  .07  for  B  =  100.  Table  4.1  displays  CV{aB}  for  various  choices 
of  B  and  CV{a},  assuming  that  E5  —  0  as  in  Efron  (1987). 
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Table  4.1:CV{aB},  as  a  function  of  B,  and  CV{a} 


CV{a} 

50 

B 
100 

-» 
200 

oo 

.25 

.27 

.26 

.25 

.25 

.20 

.22 

.21 

.21 

.20 

.15 

.18 

.17 

.16 

.15 

.10 

.14 

.12 

.11 

.10 

.05 

.11 

.09 

.07 

.05 

.00 

.10 

.07 

.05 

.00 

Prom  Table  4.1,  we  note  that  the  values  of  CV{gb]  do  not  really  change  signif- 
icantly by  increasing  the  value  of  B  from  100  to  oo.  Thereby,  Efron  (1987,  p.  200) 
commented  that  "there  is  little  improvement  past  B  =  100."  His  rule  of  thumb  says 
that  B  as  small  as  50  is  good  enough  to  get  a  reasonable  bootstrap  estimate  of  a. 

However,  we  take  the  view  that  bootstrap  standard  errors  will  not  be  used  in  prac- 
tice unless  the  same  values  are  consistently  obtained  when  the  technique  is  repeatedly 
applied  to  the  same  data.  In  other  words,  the  value  of  B  should  be  chosen  so  that 
the  Monte  Carlo  error  in  bB  is  negligible.  Efron's  criterion  fails  to  accomplish  this 
objective  because  it  is  based  on  the  total  error,  including  both  sampling  variability 
and  simulation  components. 

When  k  =  1,  our  approach  is  to  find  B  such  that  the  inequality 

1-S<^<1  +  S, 

holds,  with  high  probability  for  some  very  small  5  (>  0).  This  approach  can  be 
generalized  to  the  case  k  >  1  as  follows.  Notice  that  the  bootstrap  estimate  of 
variance  of  the  linear  combination,  aT/3,  is  aTS  a,  with  corresponding  Monte  Carlo 
approximation  aTY,Ba.  In  general  we  will  like  aTT,Ba  to  be  within  100  x  8%  of 
aT0.  Therefore,  mimicking  the  criterion  for  the  univariate  case  we  require  B  large 
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enough  such  that 


.  aT£Ba 

1-5  <  —  <  1  +  8, 

aT£a 


for  any  a  ^  0  with  high  probability. 


4.2    Description  of  the  Method 


We  note  that  the  deviation  vectors,  d*  s,  j  =  1,  . . .,  B  are  zi.d.  conditional  on 
the  sample.  Also  if  the  sample  size,  n,  is  large  d*  follows  approximately  a  k  variate 

multivariate  normal  distribution  with  mean  zero  and  variance-covariance  matrix  E. 
That  is 

dfp£OIiVfc(0,E), 

and  hence  for  any  vector  a, 

Tj*  a-pprox 


ard*  ^    jV(0,aJ  £a), 


for  all  j  =  1,  . . . ,  B.  Since  T,B  ->  £,  it  follows  that 


aTSa 


for  all  choices  of  the  vector  a,  as  B  ->  oo.  Here  P*  denotes  the  probability  under  of 
the  fitted  distribution  F.  Equivalently 

1      (1-5)<anio"l^-SUp-?F-<(1  +  ^  H  *' 
t  a7£a      a^o  oTSa  J 

as  5  *4  oo,  for  any  <5  >  0.  Notice  that  infa^0  aT^a  =  /m  and  supn     Qr^|ftQ  = 


/(it),  where  and  l^)  are  the  largest  and  the  smallest  eigenvalues  of  £  1'EB  (An- 
derson, 1971  (Theorem  A.2.4)).  The  normality  of  the  rf*'s  implies  that  BSB  has 
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Wishartfc(E,  B)  distribution  approximately  and  hence 

B  E"1  tB  ap%>°x  Wishartk(I,  B).  (4.2.3) 

That  is,  the  joint  distribution  of  the  eigenvalues  of  E  1  EB  does  not  depend  on  E. 
When  k  =  1,  E  and  E#  are  scalars  and  we  write  E  =  a2  and  Eg  =  b\.  In  this  case 

.  ,  arEBa          aTEBa  a\ 

inr   ^ —  =  sup  x —  -  — . 

a^o  aTEa      a/o  aTEa  ^2 

Hence,  using  (4.2.3), 


P*{l-6<^  <1  +  S}  =  P*{B(l-6)<xl<B(l  +  5)}. 


Using  the  approximation  \\  ~  B  +  \/2BZ,  where  Z  is  standard  normal,  we  obtain 

^*|i-^<|f<i  +  ^}  «  pjl^-^s1/2  j. 

Thus,  for  example,  B  =  5400  resamples  are  required  for  <r|  to  be  within  5%  (£  =  .05) 
of  <t2  with  probability  0.99.  In  Table  4.2  we  tabulate  the  required  values  of  B  for 
different  combinations  of  a  and  5.  We  note  that  these  values  of  B  are  significantly 
larger  than  than  the  values  of  B  recommended  by  Efron. 

Table  4.2:Required  number  of  bootstrap  resamples  for  different  a,  8 


5 

a 

.10 

.05 

.01 

.10 

575 

775 

1400 

.05 

2175 

3100 

5400 

In  the  general  case,  where  k  >  1,  our  goal  is  to  find  B  such  that 

l-a  <  P*{1-6<1(1)  <l(k)  <1  +  S}, 


(4.2.4) 
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for  given  values  of  a  and  5.  In  order  to  proceed  in  the  general  case,  we  require  a  large 
sample  approximation  to  the  joint  distribution  of  /(i)  and  l^)-  The  following  result  is 
due  to  Anderson  (1963). 

Lemma  1  :  Let  BE  1'EB  be  distributed  according  to  Wishartk(I,  B),  and  define  the 

diagonal  matrix  L  =  Diag{l{\) . . .,  /(*)}  by  £  T>b  =  CTL  C,  CTC  =  J,  /(i)  <  /(2)  < 
•  •  •  <  /(*),  and  di  >  0,  *  =  1,  k.  Then  the  density  of  the  limiting  distribution  of 

\/B(L  -  I)  =  D  =  Diag{du  . .  .,dk},  as  B     oo  is  given  by 

t=l  z  \zi=l    /  i<j 

Using  Lemma  1,  in  principle  we  can  find  values  L  and  U  such  that 

1  -  a   <   P*  {L  <  di  <  dk  <  U} 

=  P*{L  <  VS(lw  -  1)  <  VB(l{k)  -1)<U] 


p^7$  +  1<lw<l{k)<7B=  +  1\  (4-2-5) 


for  large  B.  Comparing  (4.2.4)  and  (4.2.5)  we  obtain 

L  =  -tfv^  and  f/  =  +5\/B. 
It  follows  that  L  —  —U  and  hence, 


B  =  (^)2.  (4.2.6) 


In  practice  however,  it  is  very  difficult  to  calculate  the  probability  in  (4.2.5)  using 
the  density  function  in  Lemma  1.  The  problem  of  integrating  this  density  function  is 
simplified  by  using  the  following  theorem. 
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Theorem  1.  If  the  joint  distribution  of  d\,  . .  .dk  is  as  given  in  (4),  then 


P*  [-U  <dl<dk<U]  =  s{k)  x  p(k,  -U,  U) 


(4.2.7) 


where 


p{2m,-U,+U)  if  k  =  2m 

p(k,-U,U)  =  1{  (4.2.8) 
afe<-l)<J»i(£0C^2m  + 1,  -U,  U)  if  k  =  2m  +  l 


and 


p(2m,  -U,  U)  =  2r 


Ft 

S 


Fx  F3 

r2m-2  r2m-2 


rr<2m.—  1 
•  *0 


I7i2m-1 
r2m-2 


Gt(2m  +  l,-U,U)  =  K/Ci1)^!  t-i,t+i,..2m+i\1/2 

Fls  =  f  Fs{e)ete-92'2de 

Fs(9)=  f  e'^x'dx 

J  —  U 

and     /*  =  0  or  2FSJ   if  s  + 1  is  even  or  odd  respectively. 

Theorem  1,  is  a  special  case  of  a  result  due  to  Mehta  (1960).  An  outline  of 
Mehta's  proof  in  this  setting  is  given  in  Section  3.  As  we  have  already  noted,  in  our 
case  U  =  S\/B.  So  for  different  values  of  k,  and  for  given  S  and  a,  one  can  calculate 
B  using  (4.2.7)  and  (4.2.6).  In  practice  this  process  requires  an  iterative  procedure. 
For  given  S  we  plug  in  a  value  B  in  (4.2.7)  and  calculate  the  probability  using  (4.2.8) 
and  check  how  close  it  is  to  1  —  a.  At  each  iteration  we  increase  the  value  of  B  until 
the  calculated  probability  is  very  close  to  1  -  a.  The  Figure  1  shows  the  values  of 
B  required,  for  different  values  of  the  triplet  (k,a,S).  We  note  that,  if  S  =  .10  our 
estimated  standard  errors  will  be  uniformly  within  roughly  5%  of  the  true  standard 
errors  with  probability  1  -  a.  In  Figure  1,  k  and  B  are  plotted  on  the  horizontal  and 
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Figure  4.1.  Number  of  Bootstrap  required 


vertical  axes  respectively.  Also  the  relationship  between  B  and  k  is  approximately 
linear  with  slope  decreasing  as  a  function  of  5.  On  the  other  hand,  the  values  of  a 
mainly  affects  the  intercept. 

The  fitted  regression  lines  in  Table  4.3  can  be  used  to  develop  rules  of  thumb  for 
evaluating  B.  For  example,  with  8  =  .1  and  a  =  .05  we  require  B  »  400  +  500k. 
That  is,  for  k  =  1  with  B  m  900,  &b  will  be  5%  of  the  true  bootstrap  estimate  a. 
Similarly,  for  k  =  2  we  require  approximately  1400  bootstrap  resamples  to  make  all 
the  four  elements  of  EB  and  all  their  linear  combinations  to  be  simultaneously  within 
10%  of  the  elements  and  the  linear  combinations  of  S  respectively. 
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Table  4.3:Required  number  of  bootstrap  resamples  for  different  a,  S  and  k 


5 

a 

B 

.10 

.01 

952  +  518  x  k 

.10 

.05 

388  +  473  x  k 

.10 

.10 

150  +  451  x  k 

.05 

.10 

185  +  1946  x  k 

.05 

.05 

1488  +  1905  x  k 

4.3    Proof  of  Theorem  1 


Using  Lemma  1,  we  have 
P*[-U  <  di  <  dk  <  U)  = 


2^(^)/4ji^(i±i=i>/ 
t=i  1 


u<d1<dk<u       \2  i=1    J  iKj 


(4.3.9) 

M.  L.  Mehta  (1960),  develops  a  methodology  for  integrals  of  the  following  form 

p(p,k,r,L,U)=  [  e-W+-"^\0l...eky]l(0i-0j)i[dBi,  (4.3.10) 
jR  i<j  i=i 

where  the  region  of  integration  R  is  L  <  $i  <  92...<  9k  <  U,  p,  k  are  positive  integers 
and  r  is  positive  or  zero.  Mehta  (1960)  notes  that  the  method  becomes  much  simpler 
when  p  is  even  and  L  =  —U  which  is  the  case  in  our  problem,  since  comparing  (4.3.9) 
and  (4.3.10)  we  see  that,  in  our  case  p  =  2,  r  =  0  and  L  =  —U.  The  following  is  an 
outline  of  Mehta's  proof. 

The  integrand  in  (4.3.10)  could  be  written  in  the  form  of  a  determinant 


p(p,k,r,L,U)=  [  e-^+  '  +^ 
Jr. 


r+k-l 


0Vl 


0r+fc-l 


\d$i  (4.3.11) 


t=i 
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After  integrating  over  the  variable  with  odd  indices,  the  remaining  integrand  becomes 
a  symmetric  function  of  the  variables  62,64,  . ..  For  k  =  2m  (4.3.11)  can  be  written 
as 


p(p,2m,r,L,U)  =  I  e"^** 
Jr 


61 


Fr(62) 

Fr+\(62)  62 


(62)  6r2 


r+2m-l 


■  •  02m 

nr+1 

■  ■  y2m 


■  ■  y2m 


(4.3.12) 


and  for  k  =  2m  +  1 


p(p,2m+  \,r,L,U)  =  I  e_5Xi^ 

Jr 


Fr(62)       6r2       ...  Fr(U) 
Fr+i(62)     6r2+1     ...  Fr+1(U) 


i=l 


rr+2m 

(62)  6r+2m  ...  Fr+2m(U) 

(4.3.13) 

The  symmetry  of  the  integrand  and  the  independence  of  the  variables  in  (4.3.12)  lead 
to  the  expression 


p(p,  2m,  r,  L,  U)  = 

0 

fr+l 

frr+l  • 

0 

jr+2m-l 

/r+2m-l 
•  •  Jr+1 

fr+2m-l 

fr+l 

Jr+2m-\  • 

..  0 

1/2 


(4.3.14) 


where  the  numbers  f\  are  given  by 


ft  _  pt  _  ps 
Js       rs  rti 


and  where 


s  JL<x 


x'd'dxdd. 


<e<u 


If  p  is  even  and  L  =  -U,  there  are  further  simplifications,  as  we  have  then 
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giving  //  =  0  or  2F/  depending  on  whether  s  4-  t  is  even  or  odd.  The  quantity  p  is 
then  expressible  in  the  form  of  a  determinant, 


p{p,2m,r,-U,U)  =  T 


pr+l 

jpr+l 
*r+2 


rr+2m-2  rr+2m-2 


jnr+2m-l 

r?r+2m-l 
r  r+2m-2 


(4.3.15) 


In  our  case  with  p  —  2  and  r 

=  0  we  get 

p(2,2m,0,-U,U) 

=  2m 

• 

Ft  . 

■•  ^o2m 
..  Flm 

^2771-2 

^2^1-2  • 

T?2m 
■  ■  t2m- 

(4.3.16) 


where  in  this  case 


e-^-^x'tfdxdd. 


-u<x<e<u 

For  different  values  of  s  and  t  this  F/  can  be  calculated  using  simple  numerical 
integration.  Therefore,  for  k  =  2m,  using  (4.2.6)  in  (4.2.8)  we  get 

P*[-U  <  di  <  4  <  V]  =  P* [SVB  <  di  <  dk  <  5VB] 

=  s(k)  x  p(2m,  -6y/B,  +SVB), 

where  s(k)  =  2-*/27r*(*-1)/4  UUi  I*^*1^)  and  p(-)  is  given  in  (4.3.16). 

Mehta  (1960)  did  not  provide  a  proof  of  the  case  k  —  2m  +  1,  but  states  that 
"the  case  of  odd  k  requires  only  slight  additional  care,  the  final  results  are  however 
unchanged  " .  The  following  argument  indicates  specifically  how  the  "odd"  case  can 
be  handled. 

Let  Di  denote  the  determinant  obtained  by  deleting  the  last  column  and  (i  +  1) 
th  row  in  the  determinant  on  the  right  hand  side  of  (4.3.13).  Then 

2m  .  m 

p(p,2m  +  l,r,L,U)  =  £  (-l)lFr+i(U)  /  A  Ui^^i)  (4-3-17) 

t=i  jR  j=i 
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Here  we  note  that  each  D<  is  of  even  order,  and  the  integrand  is  same  as  the  one  in 
(4.3.12).  Now  one  can  use  the  same  argument  as  used  in  k  =  2m.  Thus, 


2m 


p(p,  2m  +  1,  r,  L,U)  =  Y,  (-lYFr+i(U)Gl+1  (p,  2m  +  1,  r,  L,  U)  (4.3.18) 


i=i 


where 

Gt (p,  2m+l,r,L,U)  =  | (f^)i,j=1, ....t-M+i,..2m+i| 1/2 


and  f\  is  defined  as  before.  Combining  (4.3.16)  and  (4.3.18)  we  get  our  required 
result. 


CHAPTER  5 
SUMMARY  AND  FUTURE  RESEARCH 

5.1  Summary 

In  summary,  as  noted  in  the  previous  chapters,  one  can  improve  on  estimators 
or  predictors  of  several  local  area  means  by  incorporating  information  from  similar 
neighboring  sources.  The  proposed  empirical  Bayes  estimator  achieves  this  objective. 
The  method  is  fairly  general  in  the  sense  that  the  observations  in  each  local  area 
are  considered  to  be  realizations  from  a  superpopulation  of  one  parameter  exponen- 
tial family  distributions  with  quadratic  variance  function.  Thus  our  empirical  Bayes 
model  includes  the  normal,  binomial,  Poisson,  gamma  and  negative  binomial  super- 
populations.  The  Bayes  risks  of  these  empirical  Bayes  estimators  of  the  local  area 
means  are  obtained,  and  asymptotic  optimality  properties  of  these  empirical  Bayes 
estimators  are  proved.  We  also  provide  a  methodology  to  find  the  mean  square  errors 
and  the  confidence  intervals  of  the  proposed  empirical  Bayes  estimators.  The  para- 
metric bootstrap  procedure  is  used  for  this  purpose.  Also,  provided  is  some  frequen- 
tist  justification  of  empirical  Bayes  conditional  and  unconditional  confidence  inter- 
vals. Mathematical  results  are  proved  to  compare  the  conditional  and  unconditional 
asymptotic  distributions  of  the  standardized  empirical  Bayes  estimators.  Asymptotic 
methods  are  used  to  develop  rules-of-thumb  for  the  number  of  resamples  required  for 
accurate  Monte  Carlo  approximation  of  bootstrap  variance  covariance  matrix. 
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5.2    Future  Research 

In  this  dissertation,  we  considered  the  problem  of  point  and  interval  estimation 
of  local  area  means,  where  small  samples  are  available  from  a  moderate  to  large 
number  of  locations,  and  estimates  for  different  locations  are  required.  The  solution 
provided  in  this  dissertation  is  an  empirical  Bayes  one.  Another  way  of  addressing  this 
small  area  problem  is  to  use  generalized  linear  mixed  models  (GLMMs),  an  extension 
of  generalized  linear  models  (GLMs).  In  GLMMs  dependence  among  outcomes  is 
modelled  by  introducing  random  effects  in  the  linear  predictor.  The  GLMMs  can 
also  be  regarded  as  extensions  of  the  general  mixed  effects  linear  models  in  the  same 
fashion  as  the  GLMs  extend  the  classical  fixed  effect  linear  models. 

To  establish  notation  in  the  context  of  small  area  problem,  let  us  suppose  data 
is  collected  from  k  "locations".  Let  yji  denote  the  zth  response  at  location  j,  i  = 
1,  . . .  ,rij.  Let  Xj  denote  a  p- vector  of  known  covariate  values  associated  with  sample 
mean  y~j,  and  let  u  =  (t*i,  . . . ,  Uf.)T  be  a  vector  of  unknown  location  effect. 

Suppose  that,  conditional  on  the  location  effects,  the  responses  come  from  a  GLM 
with  linear  predictors 

Vj  =  xfP  +  Uj, 

where  (3  is  an  unknown  vector  of  fixed  effects,  and  means  fij  =  E(y~j\u)  satisfying 
g(Hj)  =  rjj,  for  some  link  function  g.  In  the  notation  of  McCullagh  and  Nelder  (1989), 
the  responses  are  (conditionally)  independent  with  joint  density  functions  of  the  form 

MfjfaAt)  =  exp^iVjOj  -  &$))  +  c{yj,4>/wj)^,  (5.2.1) 

where  the  tfl/s  are  known  weights  and  the  mean  and  canonical  parameters  are  related 
through  the  equation  //  =  b'(9).  Suppose  our  concern  is  the  estimation/prediction  of 
linear  combinations  of  the  form  rjj  =  xj(3  +  Uj. 
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Different  assumptions  about  the  location  effects  lead  to  different  estimates  of 
Several  approximate  likelihood  methods  have  recently  been  proposed  for  parameter 
estimation  in  GLMMs  (see  Goldstein,  1991;  Schall,  1991;  Breslow  and  Clayton,  1993). 
Thus  using  any  of  these  methods  one  can  obtain  the  predictions  f)j  =  xJ(3  +  Uj  arising 
from  a  mixed  model,  in  which  the  location  effects  are  assumed  to  be  iid  normal  variates 
with  zero  mean  and  unknown  variance,  a\.  This  GLMM  model  is  the  simplest  and 
by  far  the  most  common  in  our  experience. 

The  standard  errors  of  prediction  are  usually  obtained  as  the  square  root  of  the 
unconditional  mean  squared  error  of  prediction  (UMSEP).  Booth  and  Hobert  (1996) 
present  a  pair  of  fundamental  axioms  of  prediction  that  suggest  that  the  unconditional 
approach  is  inappropriate  outside  the  normal  case.  They  suggest  that  standard  errors 
of  prediction  for  the  jth  linear  predictor  be  obtained  as  the  square  root  of  v\ (/3;  yj), 
the  conditional  mean  squared  error  of  prediction  (CMSEP),  given  by 

SMyJ^Eiivj-vtflyj},  (5-2.2) 

instead  of  the  square  root  of  the  unconditional  mean  squared  error  of  prediction 
(UMSEP)  E{vl{f}\yj)}. 

Instead  of  focussing  on  mean  squared  error,  one  could  approximate  the  asymptotic 
distribution  of 

for  large  nj  and  k.  This  will  show,  if  at  least  to  the  first  order,  the  conditional  and 
the  unconditional  GLMM  inferences  are  the  same  or  not.  We  expect  these  findings 
to  be  similar  to  those  in  Chapter  3  of  this  dissertation.  However,  estimation  of  the 
entire  distribution  in  (5.2.3)  allows  for  more  precise  inferences  about  r]j. 
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